[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568607354921-12282-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 21:03:27 dist-test-slave-vpvm krb5kdc[12294](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 21:03:27 dist-test-slave-vpvm krb5kdc[12294](info): set up 2 sockets
Aug 12 21:03:27 dist-test-slave-vpvm krb5kdc[12294](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260812 21:03:28.816304 12282 mini_kdc.cc:121] Time spent starting KDC: real 1.429s user 0.006s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568607354921-12282-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568607354921-12282-0/krb5kdc/test-user.keytab.
Aug 12 21:03:28 dist-test-slave-vpvm krb5kdc[12294](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568608, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T21:03:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T21:03:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.254.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.254.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.254.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.190.keytab.
Entry for principal kudu/127.11.254.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.190.keytab.
I20260812 21:03:28.941244 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568607354921-12282-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568607354921-12282-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568607354921-12282-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.254.190:34727
--webserver_interface=127.11.254.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.190.keytab
--principal=kudu/127.11.254.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.254.148:34791
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.254.190:34727
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568607354921-12282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568607354921-12282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568607354921-12282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:03:29.032761 12319 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:03:29.033118 12319 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:03:29.033210 12319 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:03:29.034641 12319 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 21:03:29.034700 12319 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:03:29.034723 12319 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:03:29.034742 12319 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 21:03:29.034762 12319 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 21:03:29.036706 12319 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.254.148:34791
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568607354921-12282-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568607354921-12282-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.254.190:34727
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.190.keytab
--principal=kudu/127.11.254.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.254.190:34727
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.254.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12319
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568607354921-12282-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:03:29.037083 12319 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:03:29.037369 12319 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:03:29.044653 12335 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:03:29.045127 12328 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:03:29.045156 12332 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:03:29.046149 12319 server_base.cc:1034] running on GCE node
I20260812 21:03:29.046490 12319 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:03:29.046901 12319 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:03:29.048072 12319 hybrid_clock.cc:648] HybridClock initialized: now 1786568609048028 us; error 61 us; skew 500 ppm
Aug 12 21:03:29 dist-test-slave-vpvm krb5kdc[12294](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568609, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:03:29.050796 12319 init.cc:377] Logged in from keytab as kudu/127.11.254.190@KRBTEST.COM (short username kudu)
I20260812 21:03:29.051465 12319 webserver.cc:460] Webserver started at http://127.11.254.190:36307/ using document root <none> and password file <none>
I20260812 21:03:29.051755 12319 fs_manager.cc:359] Metadata directory not provided
I20260812 21:03:29.051805 12319 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:03:29.051972 12319 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:03:29.053396 12319 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568607354921-12282-0/minicluster-data/master-0/data/instance:
uuid: "7211478f74444562a6b011d076bbd344"
format_stamp: "Formatted at 2026-08-12 21:03:29 on dist-test-slave-vpvm"
server_key: "7cd8ebd7dbc64a040e6e02f0f3cb78ad"
server_key_iv: "e379bc3720a0a5b913e82323b83d3a4c"
server_key_version: "encryptionkey@0"
I20260812 21:03:29.054150 12319 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568607354921-12282-0/minicluster-data/master-0/wal/instance:
uuid: "7211478f74444562a6b011d076bbd344"
format_stamp: "Formatted at 2026-08-12 21:03:29 on dist-test-slave-vpvm"
server_key: "7cd8ebd7dbc64a040e6e02f0f3cb78ad"
server_key_iv: "e379bc3720a0a5b913e82323b83d3a4c"
server_key_version: "encryptionkey@0"
I20260812 21:03:29.055903 12319 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.002s
I20260812 21:03:29.056937 12348 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:03:29.057205 12319 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 21:03:29.057293 12319 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568607354921-12282-0/minicluster-data/master-0/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568607354921-12282-0/minicluster-data/master-0/wal
uuid: "7211478f74444562a6b011d076bbd344"
format_stamp: "Formatted at 2026-08-12 21:03:29 on dist-test-slave-vpvm"
server_key: "7cd8ebd7dbc64a040e6e02f0f3cb78ad"
server_key_iv: "e379bc3720a0a5b913e82323b83d3a4c"
server_key_version: "encryptionkey@0"
I20260812 21:03:29.057353 12319 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568607354921-12282-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568607354921-12282-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568607354921-12282-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:03:29.076651 12319 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:03:29.079089 12319 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:03:29.079274 12319 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:03:29.084206 12319 rpc_server.cc:307] RPC server started. Bound to: 127.11.254.190:34727
I20260812 21:03:29.084306 12439 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.254.190:34727 every 8 connection(s)
I20260812 21:03:29.084870 12319 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
I20260812 21:03:29.086470 12440 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:03:29.087411 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 12319
I20260812 21:03:29.087499 12282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568607354921-12282-0/minicluster-data/master-0/wal/instance
I20260812 21:03:29.088160 12282 external_mini_cluster.cc:1428] Setting key 56f2c1fdf1ec602e244428dad9e15287
I20260812 21:03:29.090193 12440 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7211478f74444562a6b011d076bbd344: Bootstrap starting.
I20260812 21:03:29.091145 12440 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7211478f74444562a6b011d076bbd344: Neither blocks nor log segments found. Creating new log.
I20260812 21:03:29.091511 12440 log.cc:824] T 00000000000000000000000000000000 P 7211478f74444562a6b011d076bbd344: Log is configured to *not* fsync() on all Append() calls
I20260812 21:03:29.092554 12440 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7211478f74444562a6b011d076bbd344: No bootstrap required, opened a new log
I20260812 21:03:29.095461 12440 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7211478f74444562a6b011d076bbd344 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7211478f74444562a6b011d076bbd344" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 34727 } }
I20260812 21:03:29.095793 12440 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7211478f74444562a6b011d076bbd344 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:03:29.096007 12440 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7211478f74444562a6b011d076bbd344 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7211478f74444562a6b011d076bbd344, State: Initialized, Role: FOLLOWER
I20260812 21:03:29.096167 12440 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7211478f74444562a6b011d076bbd344 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7211478f74444562a6b011d076bbd344" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 34727 } }
I20260812 21:03:29.096284 12440 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7211478f74444562a6b011d076bbd344 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 21:03:29.096325 12440 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7211478f74444562a6b011d076bbd344 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 21:03:29.096378 12440 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7211478f74444562a6b011d076bbd344 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:03:29.096999 12440 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7211478f74444562a6b011d076bbd344 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7211478f74444562a6b011d076bbd344" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 34727 } }
I20260812 21:03:29.097214 12440 leader_election.cc:302] T 00000000000000000000000000000000 P 7211478f74444562a6b011d076bbd344 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7211478f74444562a6b011d076bbd344; no voters:
I20260812 21:03:29.097504 12440 leader_election.cc:288] T 00000000000000000000000000000000 P 7211478f74444562a6b011d076bbd344 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 21:03:29.097575 12448 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7211478f74444562a6b011d076bbd344 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:03:29.097764 12448 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7211478f74444562a6b011d076bbd344 [term 1 LEADER]: Becoming Leader. State: Replica: 7211478f74444562a6b011d076bbd344, State: Running, Role: LEADER
I20260812 21:03:29.097975 12448 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7211478f74444562a6b011d076bbd344 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7211478f74444562a6b011d076bbd344" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 34727 } }
I20260812 21:03:29.098110 12440 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7211478f74444562a6b011d076bbd344 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 21:03:29.098538 12448 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7211478f74444562a6b011d076bbd344 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7211478f74444562a6b011d076bbd344. Latest consensus state: current_term: 1 leader_uuid: "7211478f74444562a6b011d076bbd344" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7211478f74444562a6b011d076bbd344" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 34727 } } }
I20260812 21:03:29.098973 12448 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7211478f74444562a6b011d076bbd344 [sys.catalog]: This master's current role is: LEADER
I20260812 21:03:29.098894 12451 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7211478f74444562a6b011d076bbd344 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7211478f74444562a6b011d076bbd344" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7211478f74444562a6b011d076bbd344" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 34727 } } }
I20260812 21:03:29.099434 12451 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7211478f74444562a6b011d076bbd344 [sys.catalog]: This master's current role is: LEADER
W20260812 21:03:29.100867 12475 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7211478f74444562a6b011d076bbd344: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 21:03:29.100929 12475 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 21:03:29.100991 12462 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 21:03:29.101377 12462 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 12 21:03:29 dist-test-slave-vpvm krb5kdc[12294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568608, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.254.190@KRBTEST.COM
I20260812 21:03:29.103878 12462 catalog_manager.cc:1347] Generated new cluster ID: 3aac63e292f94ae4bcbf1a944028697a
I20260812 21:03:29.103940 12462 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 21:03:29.106447 12444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:29.089320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42800 (local address 127.11.254.190:34727)
0812 21:03:29.089701 (+ 381us) server_negotiation.cc:206] Beginning negotiation
0812 21:03:29.089712 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 21:03:29.089880 (+ 168us) server_negotiation.cc:407] Connection header received
0812 21:03:29.092137 (+ 2257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:03:29.092145 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:03:29.092248 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:03:29.092398 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:03:29.094241 (+ 1843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:29.095777 (+ 1536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:29.097035 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:29.097670 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:29.100270 (+ 2600us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:29.102556 (+ 2286us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:03:29.102561 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:03:29.102562 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:03:29.102576 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:03:29.104511 (+ 1935us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:29.104945 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:29.104948 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:29.104948 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:29.105044 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:29.105355 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:29.105356 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:29.105356 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:29.105606 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:03:29.105744 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0812 21:03:29.106061 (+ 317us) server_negotiation.cc:299] Negotiation successful
0812 21:03:29.106197 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":204,"thread_start_us":107,"threads_started":1}
I20260812 21:03:29.111266 12462 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 21:03:29.111815 12462 catalog_manager.cc:1504] Loading token signing keys...
I20260812 21:03:29.121990 12462 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7211478f74444562a6b011d076bbd344: Generated new TSK 0
I20260812 21:03:29.122255 12462 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.254.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.254.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.254.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.129.keytab.
Entry for principal kudu/127.11.254.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.129.keytab.
I20260812 21:03:29.152434 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568607354921-12282-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568607354921-12282-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568607354921-12282-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568607354921-12282-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.254.129:0
--local_ip_for_outbound_sockets=127.11.254.129
--webserver_interface=127.11.254.129
--webserver_port=0
--tserver_master_addrs=127.11.254.190:34727
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.129.keytab
--principal=kudu/127.11.254.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.254.148:34791
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568607354921-12282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568607354921-12282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568607354921-12282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:03:29.275251 12487 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:03:29.275516 12487 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:03:29.275547 12487 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:03:29.276908 12487 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:03:29.276971 12487 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:03:29.277005 12487 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.254.129
I20260812 21:03:29.278961 12487 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.254.148:34791
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568607354921-12282-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568607354921-12282-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.129.keytab
--principal=kudu/127.11.254.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.254.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568607354921-12282-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.254.129
--webserver_port=0
--tserver_master_addrs=127.11.254.190:34727
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12487
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.254.129
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568607354921-12282-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:03:29.279330 12487 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:03:29.279599 12487 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:03:29.286406 12499 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:03:29.286397 12498 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:03:29.286726 12503 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:03:29.287003 12487 server_base.cc:1034] running on GCE node
I20260812 21:03:29.287240 12487 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:03:29.287534 12487 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:03:29.288698 12487 hybrid_clock.cc:648] HybridClock initialized: now 1786568609288677 us; error 35 us; skew 500 ppm
Aug 12 21:03:29 dist-test-slave-vpvm krb5kdc[12294](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568609, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:03:29.297122 12487 init.cc:377] Logged in from keytab as kudu/127.11.254.129@KRBTEST.COM (short username kudu)
I20260812 21:03:29.297761 12487 webserver.cc:460] Webserver started at http://127.11.254.129:44647/ using document root <none> and password file <none>
I20260812 21:03:29.298067 12487 fs_manager.cc:359] Metadata directory not provided
I20260812 21:03:29.298127 12487 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:03:29.298256 12487 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:03:29.299697 12487 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568607354921-12282-0/minicluster-data/ts-0/data/instance:
uuid: "1a77bf53e1344ea3b127ca8a22b5cad6"
format_stamp: "Formatted at 2026-08-12 21:03:29 on dist-test-slave-vpvm"
server_key: "870626549dd9ac20274e30093999824e"
server_key_iv: "48b5dda4010f19eb1e22e776066420dd"
server_key_version: "encryptionkey@0"
I20260812 21:03:29.300182 12487 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568607354921-12282-0/minicluster-data/ts-0/wal/instance:
uuid: "1a77bf53e1344ea3b127ca8a22b5cad6"
format_stamp: "Formatted at 2026-08-12 21:03:29 on dist-test-slave-vpvm"
server_key: "870626549dd9ac20274e30093999824e"
server_key_iv: "48b5dda4010f19eb1e22e776066420dd"
server_key_version: "encryptionkey@0"
I20260812 21:03:29.301869 12487 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.004s
I20260812 21:03:29.302899 12514 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:03:29.303077 12487 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 21:03:29.303172 12487 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568607354921-12282-0/minicluster-data/ts-0/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568607354921-12282-0/minicluster-data/ts-0/wal
uuid: "1a77bf53e1344ea3b127ca8a22b5cad6"
format_stamp: "Formatted at 2026-08-12 21:03:29 on dist-test-slave-vpvm"
server_key: "870626549dd9ac20274e30093999824e"
server_key_iv: "48b5dda4010f19eb1e22e776066420dd"
server_key_version: "encryptionkey@0"
I20260812 21:03:29.303249 12487 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568607354921-12282-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568607354921-12282-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568607354921-12282-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:03:29.312191 12487 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:03:29.314337 12487 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:03:29.314472 12487 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:03:29.314832 12487 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:03:29.315227 12487 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:03:29.315302 12487 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:03:29.315330 12487 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:03:29.315352 12487 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:03:29.329584 12487 rpc_server.cc:307] RPC server started. Bound to: 127.11.254.129:45861
I20260812 21:03:29.329736 12695 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.254.129:45861 every 8 connection(s)
I20260812 21:03:29.330123 12487 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568607354921-12282-0/minicluster-data/ts-0/data/info.pb
Aug 12 21:03:29 dist-test-slave-vpvm krb5kdc[12294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568609, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.129@KRBTEST.COM for kudu/127.11.254.190@KRBTEST.COM
I20260812 21:03:29.339313 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 12487
I20260812 21:03:29.339396 12282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568607354921-12282-0/minicluster-data/ts-0/wal/instance
I20260812 21:03:29.339531 12282 external_mini_cluster.cc:1428] Setting key ad2c0c7eb7f3860a0d641a2313b3a864
I20260812 21:03:29.341171 12444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:29.331190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.129:33995 (local address 127.11.254.190:34727)
0812 21:03:29.331245 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0812 21:03:29.331247 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:03:29.331640 (+ 393us) server_negotiation.cc:407] Connection header received
0812 21:03:29.332438 (+ 798us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:03:29.332439 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:03:29.332451 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:03:29.332510 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:03:29.333781 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:29.334330 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:29.335563 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:29.335799 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:29.335864 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:29.338355 (+ 2491us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:03:29.338357 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:03:29.338357 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:03:29.338365 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:03:29.339749 (+ 1384us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:29.340214 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:29.340215 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:29.340215 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:29.340276 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:29.340555 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:29.340556 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:29.340557 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:29.340667 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:03:29.340720 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0812 21:03:29.341032 (+ 312us) server_negotiation.cc:299] Negotiation successful
0812 21:03:29.341103 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260812 21:03:29.341480 12701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:29.331155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:34727 (local address 127.11.254.129:33995)
0812 21:03:29.331460 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0812 21:03:29.331508 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 21:03:29.332326 (+ 818us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:03:29.332569 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:03:29.332573 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:03:29.333361 (+ 788us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:03:29.333673 (+ 312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:29.333678 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:29.334440 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:29.334441 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:29.335463 (+ 1022us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:29.335466 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:29.335870 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:29.335870 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:29.336018 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:29.336681 (+ 663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:03:29.336690 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:03:29.338261 (+ 1571us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:03:29.339885 (+ 1624us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:03:29.339891 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:03:29.339893 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:03:29.340149 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:03:29.340337 (+ 188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:03:29.340337 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:03:29.340338 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:03:29.340482 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:03:29.340725 (+ 243us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:03:29.340731 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:03:29.340898 (+ 167us) client_negotiation.cc:769] Sending connection context
0812 21:03:29.341007 (+ 109us) client_negotiation.cc:240] Negotiation successful
0812 21:03:29.341188 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":156,"threads_started":1}
I20260812 21:03:29.342121 12697 heartbeater.cc:347] Connected to a master server at 127.11.254.190:34727
I20260812 21:03:29.342238 12697 heartbeater.cc:464] Registering TS with master...
I20260812 21:03:29.342449 12697 heartbeater.cc:511] Master 127.11.254.190:34727 requested a full tablet report, sending...
I20260812 21:03:29.343401 12384 ts_manager.cc:194] Registered new tserver with Master: 1a77bf53e1344ea3b127ca8a22b5cad6 (127.11.254.129:45861)
I20260812 21:03:29.344264 12384 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.254.129@KRBTEST.COM'} at 127.11.254.129:33995
WARNING: no policy specified for kudu/127.11.254.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.254.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.254.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.130.keytab.
Entry for principal kudu/127.11.254.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.130.keytab.
I20260812 21:03:29.366567 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568607354921-12282-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568607354921-12282-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568607354921-12282-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568607354921-12282-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.254.130:0
--local_ip_for_outbound_sockets=127.11.254.130
--webserver_interface=127.11.254.130
--webserver_port=0
--tserver_master_addrs=127.11.254.190:34727
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.130.keytab
--principal=kudu/127.11.254.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.254.148:34791
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568607354921-12282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568607354921-12282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568607354921-12282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:03:29.483893 12705 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:03:29.484160 12705 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:03:29.484203 12705 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:03:29.485680 12705 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:03:29.485749 12705 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:03:29.485792 12705 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.254.130
I20260812 21:03:29.487668 12705 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.254.148:34791
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568607354921-12282-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568607354921-12282-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.130.keytab
--principal=kudu/127.11.254.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.254.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568607354921-12282-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.254.130
--webserver_port=0
--tserver_master_addrs=127.11.254.190:34727
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12705
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.254.130
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568607354921-12282-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:03:29.487960 12705 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:03:29.488222 12705 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:03:29.494851 12715 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:03:29.494933 12713 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:03:29.494890 12712 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:03:29.495385 12705 server_base.cc:1034] running on GCE node
I20260812 21:03:29.495687 12705 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:03:29.495998 12705 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:03:29.497157 12705 hybrid_clock.cc:648] HybridClock initialized: now 1786568609497138 us; error 37 us; skew 500 ppm
Aug 12 21:03:29 dist-test-slave-vpvm krb5kdc[12294](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568609, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:03:29.500349 12705 init.cc:377] Logged in from keytab as kudu/127.11.254.130@KRBTEST.COM (short username kudu)
I20260812 21:03:29.500816 12705 webserver.cc:460] Webserver started at http://127.11.254.130:39293/ using document root <none> and password file <none>
I20260812 21:03:29.501022 12705 fs_manager.cc:359] Metadata directory not provided
I20260812 21:03:29.501072 12705 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:03:29.501204 12705 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:03:29.502540 12705 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568607354921-12282-0/minicluster-data/ts-1/data/instance:
uuid: "e2d330dfcf9b4be2ac4c56fae53c4616"
format_stamp: "Formatted at 2026-08-12 21:03:29 on dist-test-slave-vpvm"
server_key: "d31e0a7040edb56e7a07090b982d1774"
server_key_iv: "9db286cd399e5b30303a7b026dbcc258"
server_key_version: "encryptionkey@0"
I20260812 21:03:29.502942 12705 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568607354921-12282-0/minicluster-data/ts-1/wal/instance:
uuid: "e2d330dfcf9b4be2ac4c56fae53c4616"
format_stamp: "Formatted at 2026-08-12 21:03:29 on dist-test-slave-vpvm"
server_key: "d31e0a7040edb56e7a07090b982d1774"
server_key_iv: "9db286cd399e5b30303a7b026dbcc258"
server_key_version: "encryptionkey@0"
I20260812 21:03:29.504535 12705 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 21:03:29.505282 12729 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:03:29.505483 12705 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 21:03:29.505576 12705 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568607354921-12282-0/minicluster-data/ts-1/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568607354921-12282-0/minicluster-data/ts-1/wal
uuid: "e2d330dfcf9b4be2ac4c56fae53c4616"
format_stamp: "Formatted at 2026-08-12 21:03:29 on dist-test-slave-vpvm"
server_key: "d31e0a7040edb56e7a07090b982d1774"
server_key_iv: "9db286cd399e5b30303a7b026dbcc258"
server_key_version: "encryptionkey@0"
I20260812 21:03:29.505640 12705 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568607354921-12282-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568607354921-12282-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568607354921-12282-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:03:29.529716 12705 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:03:29.532225 12705 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:03:29.532451 12705 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:03:29.532881 12705 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:03:29.533265 12705 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:03:29.533304 12705 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:03:29.533334 12705 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:03:29.533350 12705 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:03:29.548421 12705 rpc_server.cc:307] RPC server started. Bound to: 127.11.254.130:42853
I20260812 21:03:29.548648 12945 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.254.130:42853 every 8 connection(s)
I20260812 21:03:29.548928 12705 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568607354921-12282-0/minicluster-data/ts-1/data/info.pb
I20260812 21:03:29.553120 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 12705
I20260812 21:03:29.553210 12282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568607354921-12282-0/minicluster-data/ts-1/wal/instance
I20260812 21:03:29.553349 12282 external_mini_cluster.cc:1428] Setting key f934205a6ac79f44502d2321b2073d5e
Aug 12 21:03:29 dist-test-slave-vpvm krb5kdc[12294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568609, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.130@KRBTEST.COM for kudu/127.11.254.190@KRBTEST.COM
I20260812 21:03:29.559944 12444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:29.549803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.130:49809 (local address 127.11.254.190:34727)
0812 21:03:29.549867 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0812 21:03:29.549869 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:03:29.550315 (+ 446us) server_negotiation.cc:407] Connection header received
0812 21:03:29.551078 (+ 763us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:03:29.551079 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:03:29.551091 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:03:29.551156 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:03:29.552254 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:29.552847 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:29.554261 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:29.554495 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:29.554565 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:29.557180 (+ 2615us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:03:29.557182 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:03:29.557182 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:03:29.557190 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:03:29.558601 (+ 1411us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:29.558977 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:29.558978 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:29.558979 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:29.559042 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:29.559251 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:29.559252 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:29.559252 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:29.559398 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:03:29.559472 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0812 21:03:29.559752 (+ 280us) server_negotiation.cc:299] Negotiation successful
0812 21:03:29.559842 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260812 21:03:29.560169 12952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:29.549784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:34727 (local address 127.11.254.130:49809)
0812 21:03:29.550134 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0812 21:03:29.550181 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 21:03:29.550980 (+ 799us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:03:29.551226 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:03:29.551230 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:03:29.551838 (+ 608us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:03:29.552156 (+ 318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:29.552162 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:29.552925 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:29.552926 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:29.554182 (+ 1256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:29.554185 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:29.554571 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:29.554571 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:29.554746 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:29.555351 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:03:29.555360 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:03:29.557090 (+ 1730us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:03:29.558686 (+ 1596us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:03:29.558692 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:03:29.558694 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:03:29.558915 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:03:29.559103 (+ 188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:03:29.559104 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:03:29.559104 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:03:29.559194 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:03:29.559484 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:03:29.559491 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:03:29.559626 (+ 135us) client_negotiation.cc:769] Sending connection context
0812 21:03:29.559715 (+ 89us) client_negotiation.cc:240] Negotiation successful
0812 21:03:29.559849 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":211,"threads_started":1}
I20260812 21:03:29.560705 12946 heartbeater.cc:347] Connected to a master server at 127.11.254.190:34727
I20260812 21:03:29.560799 12946 heartbeater.cc:464] Registering TS with master...
I20260812 21:03:29.560992 12946 heartbeater.cc:511] Master 127.11.254.190:34727 requested a full tablet report, sending...
I20260812 21:03:29.561393 12384 ts_manager.cc:194] Registered new tserver with Master: e2d330dfcf9b4be2ac4c56fae53c4616 (127.11.254.130:42853)
I20260812 21:03:29.561903 12384 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.254.130@KRBTEST.COM'} at 127.11.254.130:49809
WARNING: no policy specified for kudu/127.11.254.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.254.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.254.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.131.keytab.
Entry for principal kudu/127.11.254.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.131.keytab.
I20260812 21:03:29.582916 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568607354921-12282-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568607354921-12282-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568607354921-12282-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568607354921-12282-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.254.131:0
--local_ip_for_outbound_sockets=127.11.254.131
--webserver_interface=127.11.254.131
--webserver_port=0
--tserver_master_addrs=127.11.254.190:34727
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.131.keytab
--principal=kudu/127.11.254.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.254.148:34791
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568607354921-12282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568607354921-12282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568607354921-12282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:03:29.705828 12958 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:03:29.706115 12958 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:03:29.706147 12958 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:03:29.707494 12958 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:03:29.707545 12958 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:03:29.707576 12958 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.254.131
I20260812 21:03:29.709389 12958 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.254.148:34791
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568607354921-12282-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568607354921-12282-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.131.keytab
--principal=kudu/127.11.254.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.254.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568607354921-12282-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.254.131
--webserver_port=0
--tserver_master_addrs=127.11.254.190:34727
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12958
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.254.131
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568607354921-12282-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:03:29.709619 12958 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:03:29.709843 12958 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:03:29.715359 12967 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:03:29.715359 12971 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:03:29.715468 12974 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:03:29.715903 12958 server_base.cc:1034] running on GCE node
I20260812 21:03:29.716100 12958 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:03:29.716317 12958 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:03:29.717463 12958 hybrid_clock.cc:648] HybridClock initialized: now 1786568609717447 us; error 30 us; skew 500 ppm
Aug 12 21:03:29 dist-test-slave-vpvm krb5kdc[12294](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568609, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:03:29.720654 12958 init.cc:377] Logged in from keytab as kudu/127.11.254.131@KRBTEST.COM (short username kudu)
I20260812 21:03:29.721109 12958 webserver.cc:460] Webserver started at http://127.11.254.131:34455/ using document root <none> and password file <none>
I20260812 21:03:29.721314 12958 fs_manager.cc:359] Metadata directory not provided
I20260812 21:03:29.721361 12958 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:03:29.721488 12958 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:03:29.722772 12958 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568607354921-12282-0/minicluster-data/ts-2/data/instance:
uuid: "d6a1fa18647748f7b844124581cef84e"
format_stamp: "Formatted at 2026-08-12 21:03:29 on dist-test-slave-vpvm"
server_key: "99c74bda322f77bb7b988048f5a6c59a"
server_key_iv: "c60a08f205ec5bcbf2f77199128da24c"
server_key_version: "encryptionkey@0"
I20260812 21:03:29.723201 12958 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568607354921-12282-0/minicluster-data/ts-2/wal/instance:
uuid: "d6a1fa18647748f7b844124581cef84e"
format_stamp: "Formatted at 2026-08-12 21:03:29 on dist-test-slave-vpvm"
server_key: "99c74bda322f77bb7b988048f5a6c59a"
server_key_iv: "c60a08f205ec5bcbf2f77199128da24c"
server_key_version: "encryptionkey@0"
I20260812 21:03:29.724844 12958 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 21:03:29.725766 12984 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:03:29.725934 12958 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 21:03:29.726063 12958 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568607354921-12282-0/minicluster-data/ts-2/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568607354921-12282-0/minicluster-data/ts-2/wal
uuid: "d6a1fa18647748f7b844124581cef84e"
format_stamp: "Formatted at 2026-08-12 21:03:29 on dist-test-slave-vpvm"
server_key: "99c74bda322f77bb7b988048f5a6c59a"
server_key_iv: "c60a08f205ec5bcbf2f77199128da24c"
server_key_version: "encryptionkey@0"
I20260812 21:03:29.726132 12958 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568607354921-12282-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568607354921-12282-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568607354921-12282-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:03:29.737627 12958 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:03:29.739936 12958 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:03:29.740131 12958 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:03:29.740511 12958 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:03:29.740867 12958 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:03:29.740904 12958 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:03:29.740933 12958 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:03:29.740959 12958 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:03:29.755191 12958 rpc_server.cc:307] RPC server started. Bound to: 127.11.254.131:43899
I20260812 21:03:29.755245 13179 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.254.131:43899 every 8 connection(s)
I20260812 21:03:29.755599 12958 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568607354921-12282-0/minicluster-data/ts-2/data/info.pb
I20260812 21:03:29.759838 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 12958
I20260812 21:03:29.760279 12282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786568607354921-12282-0/minicluster-data/ts-2/wal/instance
I20260812 21:03:29.760525 12282 external_mini_cluster.cc:1428] Setting key b3ed61f018055d9151b2aa62df8cefb0
Aug 12 21:03:29 dist-test-slave-vpvm krb5kdc[12294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568609, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.131@KRBTEST.COM for kudu/127.11.254.190@KRBTEST.COM
I20260812 21:03:29.767170 12444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:29.756496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.131:39929 (local address 127.11.254.190:34727)
0812 21:03:29.756561 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0812 21:03:29.756563 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:03:29.756911 (+ 348us) server_negotiation.cc:407] Connection header received
0812 21:03:29.757395 (+ 484us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:03:29.757397 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:03:29.757408 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:03:29.757463 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:03:29.758734 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:29.759260 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:29.760984 (+ 1724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:29.761315 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:29.761474 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:29.764075 (+ 2601us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:03:29.764077 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:03:29.764077 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:03:29.764086 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:03:29.765614 (+ 1528us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:29.766192 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:29.766193 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:29.766194 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:29.766256 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:29.766489 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:29.766490 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:29.766490 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:29.766612 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:03:29.766661 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0812 21:03:29.767018 (+ 357us) server_negotiation.cc:299] Negotiation successful
0812 21:03:29.767096 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260812 21:03:29.767369 13185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:29.756476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:34727 (local address 127.11.254.131:39929)
0812 21:03:29.756778 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0812 21:03:29.756820 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 21:03:29.757311 (+ 491us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:03:29.757575 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:03:29.757578 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:03:29.758373 (+ 795us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:03:29.758619 (+ 246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:29.758627 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:29.759360 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:29.759361 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:29.760845 (+ 1484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:29.760849 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:29.761416 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:29.761417 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:29.761574 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:29.762205 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:03:29.762214 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:03:29.763934 (+ 1720us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:03:29.765726 (+ 1792us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:03:29.765734 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:03:29.765736 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:03:29.766094 (+ 358us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:03:29.766320 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:03:29.766321 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:03:29.766321 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:03:29.766434 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:03:29.766693 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:03:29.766701 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:03:29.766872 (+ 171us) client_negotiation.cc:769] Sending connection context
0812 21:03:29.766973 (+ 101us) client_negotiation.cc:240] Negotiation successful
0812 21:03:29.767128 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":218,"threads_started":1}
I20260812 21:03:29.768024 13180 heartbeater.cc:347] Connected to a master server at 127.11.254.190:34727
I20260812 21:03:29.768136 13180 heartbeater.cc:464] Registering TS with master...
I20260812 21:03:29.768327 13180 heartbeater.cc:511] Master 127.11.254.190:34727 requested a full tablet report, sending...
I20260812 21:03:29.768800 12384 ts_manager.cc:194] Registered new tserver with Master: d6a1fa18647748f7b844124581cef84e (127.11.254.131:43899)
I20260812 21:03:29.769194 12384 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.254.131@KRBTEST.COM'} at 127.11.254.131:39929
I20260812 21:03:29.773142 12282 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 12 21:03:29 dist-test-slave-vpvm krb5kdc[12294](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568609, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 21:03:29 dist-test-slave-vpvm krb5kdc[12294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568609, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.11.254.129@KRBTEST.COM
I20260812 21:03:29.800208 13193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:29.791541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47646 (local address 127.11.254.129:45861)
0812 21:03:29.791853 (+ 312us) server_negotiation.cc:206] Beginning negotiation
0812 21:03:29.791858 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 21:03:29.791870 (+ 12us) server_negotiation.cc:407] Connection header received
0812 21:03:29.791897 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:03:29.791899 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:03:29.791913 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:03:29.791990 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:03:29.792686 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:29.793499 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:29.794269 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:29.794667 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:29.794729 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:29.797174 (+ 2445us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:03:29.797182 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:03:29.797182 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:03:29.797189 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:03:29.798818 (+ 1629us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:29.799366 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:29.799369 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:29.799370 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:29.799438 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:29.799670 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:29.799671 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:29.799672 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:29.799822 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:03:29.799928 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0812 21:03:29.799985 (+ 57us) server_negotiation.cc:299] Negotiation successful
0812 21:03:29.800097 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":270,"thread_start_us":204,"threads_started":1}
W20260812 21:03:29.800705 12555 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47646
Aug 12 21:03:29 dist-test-slave-vpvm krb5kdc[12294](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568609, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 21:03:29 dist-test-slave-vpvm krb5kdc[12294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568609, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.254.129@KRBTEST.COM
I20260812 21:03:29.824887 13193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:29.817671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47658 (local address 127.11.254.129:45861)
0812 21:03:29.817729 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0812 21:03:29.817730 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 21:03:29.817784 (+ 54us) server_negotiation.cc:407] Connection header received
0812 21:03:29.817844 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:03:29.817845 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:03:29.817860 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:03:29.817911 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:03:29.818563 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:29.819120 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:29.819888 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:29.820093 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:29.820135 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:29.822638 (+ 2503us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:03:29.822639 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:03:29.822640 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:03:29.822647 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:03:29.823878 (+ 1231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:29.824226 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:29.824228 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:29.824228 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:29.824291 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:29.824438 (+ 147us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:29.824438 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:29.824438 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:29.824559 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:03:29.824701 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0812 21:03:29.824732 (+ 31us) server_negotiation.cc:299] Negotiation successful
0812 21:03:29.824807 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260812 21:03:29.825464 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 12487
I20260812 21:03:29.841401 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 12705
I20260812 21:03:29.859145 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 12958
I20260812 21:03:29.869277 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 12319
2026-08-12T21:03:29Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2521 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568607354921-12282-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 21:03:29 dist-test-slave-vpvm krb5kdc[13205](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 21:03:29 dist-test-slave-vpvm krb5kdc[13205](info): set up 2 sockets
Aug 12 21:03:29 dist-test-slave-vpvm krb5kdc[13205](info): commencing operation
krb5kdc: starting...
W20260812 21:03:31.067570 12282 mini_kdc.cc:121] Time spent starting KDC: real 1.167s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568607354921-12282-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568607354921-12282-0/krb5kdc/test-user.keytab.
Aug 12 21:03:31 dist-test-slave-vpvm krb5kdc[13205](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568611, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T21:03:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T21:03:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.254.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.254.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.254.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.190.keytab.
Entry for principal kudu/127.11.254.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.190.keytab.
I20260812 21:03:31.192239 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568607354921-12282-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568607354921-12282-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568607354921-12282-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.254.190:41945
--webserver_interface=127.11.254.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.190.keytab
--principal=kudu/127.11.254.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.254.148:40617
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.254.190:41945
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568607354921-12282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568607354921-12282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568607354921-12282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:03:31.275557 13224 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:03:31.275789 13224 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:03:31.275811 13224 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:03:31.276997 13224 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 21:03:31.277046 13224 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:03:31.277060 13224 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:03:31.277072 13224 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 21:03:31.277086 13224 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 21:03:31.278726 13224 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.254.148:40617
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568607354921-12282-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568607354921-12282-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.254.190:41945
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.190.keytab
--principal=kudu/127.11.254.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.254.190:41945
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.254.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13224
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568607354921-12282-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:03:31.279016 13224 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:03:31.279248 13224 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:03:31.284196 13230 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:03:31.284248 13233 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:03:31.284248 13240 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:03:31.284715 13224 server_base.cc:1034] running on GCE node
I20260812 21:03:31.284976 13224 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:03:31.285308 13224 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:03:31.286458 13224 hybrid_clock.cc:648] HybridClock initialized: now 1786568611286433 us; error 41 us; skew 500 ppm
Aug 12 21:03:31 dist-test-slave-vpvm krb5kdc[13205](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568611, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:03:31.289043 13224 init.cc:377] Logged in from keytab as kudu/127.11.254.190@KRBTEST.COM (short username kudu)
I20260812 21:03:31.289529 13224 webserver.cc:460] Webserver started at http://127.11.254.190:36539/ using document root <none> and password file <none>
I20260812 21:03:31.289741 13224 fs_manager.cc:359] Metadata directory not provided
I20260812 21:03:31.289788 13224 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:03:31.289914 13224 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:03:31.291059 13224 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568607354921-12282-0/minicluster-data/master-0/data/instance:
uuid: "de01f67665eb4fbb9cb1d37909996d49"
format_stamp: "Formatted at 2026-08-12 21:03:31 on dist-test-slave-vpvm"
server_key: "5bb7eff9d3641bd0c48a05d4126a03a9"
server_key_iv: "9990f0074648074ad62e009256988621"
server_key_version: "encryptionkey@0"
I20260812 21:03:31.291422 13224 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568607354921-12282-0/minicluster-data/master-0/wal/instance:
uuid: "de01f67665eb4fbb9cb1d37909996d49"
format_stamp: "Formatted at 2026-08-12 21:03:31 on dist-test-slave-vpvm"
server_key: "5bb7eff9d3641bd0c48a05d4126a03a9"
server_key_iv: "9990f0074648074ad62e009256988621"
server_key_version: "encryptionkey@0"
I20260812 21:03:31.292763 13224 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 21:03:31.293450 13249 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:03:31.293640 13224 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 21:03:31.293735 13224 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568607354921-12282-0/minicluster-data/master-0/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568607354921-12282-0/minicluster-data/master-0/wal
uuid: "de01f67665eb4fbb9cb1d37909996d49"
format_stamp: "Formatted at 2026-08-12 21:03:31 on dist-test-slave-vpvm"
server_key: "5bb7eff9d3641bd0c48a05d4126a03a9"
server_key_iv: "9990f0074648074ad62e009256988621"
server_key_version: "encryptionkey@0"
I20260812 21:03:31.293802 13224 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568607354921-12282-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568607354921-12282-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568607354921-12282-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:03:31.308627 13224 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:03:31.310629 13224 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:03:31.310757 13224 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:03:31.314716 13224 rpc_server.cc:307] RPC server started. Bound to: 127.11.254.190:41945
I20260812 21:03:31.314777 13339 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.254.190:41945 every 8 connection(s)
I20260812 21:03:31.315109 13224 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
I20260812 21:03:31.316030 13340 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:03:31.318044 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 13224
I20260812 21:03:31.318136 12282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568607354921-12282-0/minicluster-data/master-0/wal/instance
I20260812 21:03:31.318280 12282 external_mini_cluster.cc:1428] Setting key 719dc5d3f94e31faeea02ffe38402983
I20260812 21:03:31.318529 13340 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de01f67665eb4fbb9cb1d37909996d49: Bootstrap starting.
I20260812 21:03:31.319219 13340 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P de01f67665eb4fbb9cb1d37909996d49: Neither blocks nor log segments found. Creating new log.
I20260812 21:03:31.319487 13340 log.cc:824] T 00000000000000000000000000000000 P de01f67665eb4fbb9cb1d37909996d49: Log is configured to *not* fsync() on all Append() calls
I20260812 21:03:31.320153 13340 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de01f67665eb4fbb9cb1d37909996d49: No bootstrap required, opened a new log
I20260812 21:03:31.321995 13340 raft_consensus.cc:348] T 00000000000000000000000000000000 P de01f67665eb4fbb9cb1d37909996d49 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de01f67665eb4fbb9cb1d37909996d49" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 41945 } }
I20260812 21:03:31.322144 13340 raft_consensus.cc:374] T 00000000000000000000000000000000 P de01f67665eb4fbb9cb1d37909996d49 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:03:31.322180 13340 raft_consensus.cc:729] T 00000000000000000000000000000000 P de01f67665eb4fbb9cb1d37909996d49 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de01f67665eb4fbb9cb1d37909996d49, State: Initialized, Role: FOLLOWER
I20260812 21:03:31.322309 13340 consensus_queue.cc:260] T 00000000000000000000000000000000 P de01f67665eb4fbb9cb1d37909996d49 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de01f67665eb4fbb9cb1d37909996d49" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 41945 } }
I20260812 21:03:31.322383 13340 raft_consensus.cc:388] T 00000000000000000000000000000000 P de01f67665eb4fbb9cb1d37909996d49 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 21:03:31.322414 13340 raft_consensus.cc:482] T 00000000000000000000000000000000 P de01f67665eb4fbb9cb1d37909996d49 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 21:03:31.322460 13340 raft_consensus.cc:3037] T 00000000000000000000000000000000 P de01f67665eb4fbb9cb1d37909996d49 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:03:31.323009 13340 raft_consensus.cc:504] T 00000000000000000000000000000000 P de01f67665eb4fbb9cb1d37909996d49 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de01f67665eb4fbb9cb1d37909996d49" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 41945 } }
I20260812 21:03:31.323123 13340 leader_election.cc:302] T 00000000000000000000000000000000 P de01f67665eb4fbb9cb1d37909996d49 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: de01f67665eb4fbb9cb1d37909996d49; no voters:
I20260812 21:03:31.323314 13340 leader_election.cc:288] T 00000000000000000000000000000000 P de01f67665eb4fbb9cb1d37909996d49 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 21:03:31.323438 13348 raft_consensus.cc:2781] T 00000000000000000000000000000000 P de01f67665eb4fbb9cb1d37909996d49 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:03:31.323632 13340 sys_catalog.cc:564] T 00000000000000000000000000000000 P de01f67665eb4fbb9cb1d37909996d49 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 21:03:31.323670 13348 raft_consensus.cc:686] T 00000000000000000000000000000000 P de01f67665eb4fbb9cb1d37909996d49 [term 1 LEADER]: Becoming Leader. State: Replica: de01f67665eb4fbb9cb1d37909996d49, State: Running, Role: LEADER
I20260812 21:03:31.323752 13348 consensus_queue.cc:237] T 00000000000000000000000000000000 P de01f67665eb4fbb9cb1d37909996d49 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de01f67665eb4fbb9cb1d37909996d49" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 41945 } }
I20260812 21:03:31.324100 13349 sys_catalog.cc:455] T 00000000000000000000000000000000 P de01f67665eb4fbb9cb1d37909996d49 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "de01f67665eb4fbb9cb1d37909996d49" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de01f67665eb4fbb9cb1d37909996d49" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 41945 } } }
I20260812 21:03:31.324201 13349 sys_catalog.cc:458] T 00000000000000000000000000000000 P de01f67665eb4fbb9cb1d37909996d49 [sys.catalog]: This master's current role is: LEADER
I20260812 21:03:31.324360 13350 sys_catalog.cc:455] T 00000000000000000000000000000000 P de01f67665eb4fbb9cb1d37909996d49 [sys.catalog]: SysCatalogTable state changed. Reason: New leader de01f67665eb4fbb9cb1d37909996d49. Latest consensus state: current_term: 1 leader_uuid: "de01f67665eb4fbb9cb1d37909996d49" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de01f67665eb4fbb9cb1d37909996d49" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 41945 } } }
I20260812 21:03:31.324425 13350 sys_catalog.cc:458] T 00000000000000000000000000000000 P de01f67665eb4fbb9cb1d37909996d49 [sys.catalog]: This master's current role is: LEADER
I20260812 21:03:31.324543 13362 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Aug 12 21:03:31 dist-test-slave-vpvm krb5kdc[13205](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568611, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.254.190@KRBTEST.COM
I20260812 21:03:31.325279 13362 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 21:03:31.327288 13362 catalog_manager.cc:1347] Generated new cluster ID: ce2b02f24e9a4c638f543e120295fc80
I20260812 21:03:31.327350 13362 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 21:03:31.329102 13345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:31.318809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57158 (local address 127.11.254.190:41945)
0812 21:03:31.319019 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0812 21:03:31.319027 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 21:03:31.319054 (+ 27us) server_negotiation.cc:407] Connection header received
0812 21:03:31.319538 (+ 484us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:03:31.319546 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:03:31.319652 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:03:31.319786 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:03:31.320413 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:31.321331 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:31.322137 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:31.322701 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:31.322801 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:31.325758 (+ 2957us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:03:31.325763 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:03:31.325763 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:03:31.325777 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:03:31.327588 (+ 1811us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:31.327947 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:31.327950 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:31.327950 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:31.328040 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:31.328229 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:31.328230 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:31.328230 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:31.328477 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:03:31.328636 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0812 21:03:31.328739 (+ 103us) server_negotiation.cc:299] Negotiation successful
0812 21:03:31.328880 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":108,"threads_started":1}
I20260812 21:03:31.362215 13362 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 21:03:31.362838 13362 catalog_manager.cc:1504] Loading token signing keys...
I20260812 21:03:31.372668 13362 catalog_manager.cc:5868] T 00000000000000000000000000000000 P de01f67665eb4fbb9cb1d37909996d49: Generated new TSK 0
I20260812 21:03:31.372867 13362 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.254.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.254.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.254.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.129.keytab.
Entry for principal kudu/127.11.254.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.129.keytab.
I20260812 21:03:31.412256 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568607354921-12282-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568607354921-12282-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568607354921-12282-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568607354921-12282-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.254.129:0
--local_ip_for_outbound_sockets=127.11.254.129
--webserver_interface=127.11.254.129
--webserver_port=0
--tserver_master_addrs=127.11.254.190:41945
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.129.keytab
--principal=kudu/127.11.254.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.254.148:40617
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568607354921-12282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568607354921-12282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568607354921-12282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:03:31.507570 13386 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:03:31.507798 13386 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:03:31.507823 13386 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:03:31.509052 13386 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:03:31.509104 13386 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:03:31.509132 13386 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.254.129
I20260812 21:03:31.510738 13386 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.254.148:40617
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568607354921-12282-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568607354921-12282-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.129.keytab
--principal=kudu/127.11.254.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.254.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568607354921-12282-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.254.129
--webserver_port=0
--tserver_master_addrs=127.11.254.190:41945
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13386
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.254.129
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568607354921-12282-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:03:31.510995 13386 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:03:31.511242 13386 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:03:31.516230 13412 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:03:31.516299 13386 server_base.cc:1034] running on GCE node
W20260812 21:03:31.516238 13408 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:03:31.516237 13403 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:03:31.516605 13386 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:03:31.516858 13386 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:03:31.518004 13386 hybrid_clock.cc:648] HybridClock initialized: now 1786568611517986 us; error 29 us; skew 500 ppm
Aug 12 21:03:31 dist-test-slave-vpvm krb5kdc[13205](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568611, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:03:31.520502 13386 init.cc:377] Logged in from keytab as kudu/127.11.254.129@KRBTEST.COM (short username kudu)
I20260812 21:03:31.521005 13386 webserver.cc:460] Webserver started at http://127.11.254.129:46511/ using document root <none> and password file <none>
I20260812 21:03:31.521291 13386 fs_manager.cc:359] Metadata directory not provided
I20260812 21:03:31.521340 13386 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:03:31.521466 13386 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:03:31.522570 13386 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568607354921-12282-0/minicluster-data/ts-0/data/instance:
uuid: "5e25228ef9274b15a1674730bb5ae1b9"
format_stamp: "Formatted at 2026-08-12 21:03:31 on dist-test-slave-vpvm"
server_key: "9eaa08cdb253b47aab0a0c17a24a2441"
server_key_iv: "a043ca4ac118d3086543d2853654f8ae"
server_key_version: "encryptionkey@0"
I20260812 21:03:31.522898 13386 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568607354921-12282-0/minicluster-data/ts-0/wal/instance:
uuid: "5e25228ef9274b15a1674730bb5ae1b9"
format_stamp: "Formatted at 2026-08-12 21:03:31 on dist-test-slave-vpvm"
server_key: "9eaa08cdb253b47aab0a0c17a24a2441"
server_key_iv: "a043ca4ac118d3086543d2853654f8ae"
server_key_version: "encryptionkey@0"
I20260812 21:03:31.524258 13386 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 21:03:31.524989 13423 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:03:31.525137 13386 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 21:03:31.525223 13386 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568607354921-12282-0/minicluster-data/ts-0/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568607354921-12282-0/minicluster-data/ts-0/wal
uuid: "5e25228ef9274b15a1674730bb5ae1b9"
format_stamp: "Formatted at 2026-08-12 21:03:31 on dist-test-slave-vpvm"
server_key: "9eaa08cdb253b47aab0a0c17a24a2441"
server_key_iv: "a043ca4ac118d3086543d2853654f8ae"
server_key_version: "encryptionkey@0"
I20260812 21:03:31.525283 13386 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568607354921-12282-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568607354921-12282-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568607354921-12282-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:03:31.557178 13386 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:03:31.559267 13386 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:03:31.559428 13386 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:03:31.559685 13386 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:03:31.560002 13386 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:03:31.560036 13386 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:03:31.560068 13386 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:03:31.560092 13386 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:03:31.565817 13386 rpc_server.cc:307] RPC server started. Bound to: 127.11.254.129:42877
I20260812 21:03:31.565856 13608 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.254.129:42877 every 8 connection(s)
I20260812 21:03:31.566201 13386 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568607354921-12282-0/minicluster-data/ts-0/data/info.pb
I20260812 21:03:31.569128 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 13386
I20260812 21:03:31.569208 12282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568607354921-12282-0/minicluster-data/ts-0/wal/instance
I20260812 21:03:31.569353 12282 external_mini_cluster.cc:1428] Setting key b48022e798799e508120263d88600e6b
Aug 12 21:03:31 dist-test-slave-vpvm krb5kdc[13205](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568611, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.129@KRBTEST.COM for kudu/127.11.254.190@KRBTEST.COM
I20260812 21:03:31.575980 13345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:31.566965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.129:33085 (local address 127.11.254.190:41945)
0812 21:03:31.567049 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0812 21:03:31.567051 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:03:31.567206 (+ 155us) server_negotiation.cc:407] Connection header received
0812 21:03:31.568276 (+ 1070us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:03:31.568277 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:03:31.568287 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:03:31.568337 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:03:31.568931 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:31.569526 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:31.570650 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:31.570922 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:31.570995 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:31.573309 (+ 2314us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:03:31.573311 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:03:31.573311 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:03:31.573318 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:03:31.574619 (+ 1301us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:31.575071 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:31.575071 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:31.575072 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:31.575129 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:31.575374 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:31.575375 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:31.575375 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:31.575470 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:03:31.575513 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0812 21:03:31.575842 (+ 329us) server_negotiation.cc:299] Negotiation successful
0812 21:03:31.575912 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260812 21:03:31.576306 13613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:31.566926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:41945 (local address 127.11.254.129:33085)
0812 21:03:31.567073 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0812 21:03:31.567115 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 21:03:31.568184 (+ 1069us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:03:31.568402 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:03:31.568406 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:03:31.568672 (+ 266us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:03:31.568864 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:31.568869 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:31.569645 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:31.569646 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:31.570553 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:31.570556 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:31.570994 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:31.570995 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:31.571091 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:31.571574 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:03:31.571619 (+ 45us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:03:31.573206 (+ 1587us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:03:31.574697 (+ 1491us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:03:31.574705 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:03:31.574708 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:03:31.575000 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:03:31.575183 (+ 183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:03:31.575184 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:03:31.575184 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:03:31.575318 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:03:31.575525 (+ 207us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:03:31.575530 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:03:31.575712 (+ 182us) client_negotiation.cc:769] Sending connection context
0812 21:03:31.575831 (+ 119us) client_negotiation.cc:240] Negotiation successful
0812 21:03:31.576006 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":74,"threads_started":1}
I20260812 21:03:31.576900 13610 heartbeater.cc:347] Connected to a master server at 127.11.254.190:41945
I20260812 21:03:31.577008 13610 heartbeater.cc:464] Registering TS with master...
I20260812 21:03:31.577206 13610 heartbeater.cc:511] Master 127.11.254.190:41945 requested a full tablet report, sending...
I20260812 21:03:31.577735 13275 ts_manager.cc:194] Registered new tserver with Master: 5e25228ef9274b15a1674730bb5ae1b9 (127.11.254.129:42877)
WARNING: no policy specified for kudu/127.11.254.130@KRBTEST.COM; defaulting to no policy
I20260812 21:03:31.578404 13275 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.254.129@KRBTEST.COM'} at 127.11.254.129:33085
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.254.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.254.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.130.keytab.
Entry for principal kudu/127.11.254.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.130.keytab.
I20260812 21:03:31.596864 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568607354921-12282-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568607354921-12282-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568607354921-12282-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568607354921-12282-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.254.130:0
--local_ip_for_outbound_sockets=127.11.254.130
--webserver_interface=127.11.254.130
--webserver_port=0
--tserver_master_addrs=127.11.254.190:41945
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.130.keytab
--principal=kudu/127.11.254.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.254.148:40617
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568607354921-12282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568607354921-12282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568607354921-12282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:03:31.679183 13621 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:03:31.679389 13621 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:03:31.679420 13621 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:03:31.680540 13621 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:03:31.680583 13621 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:03:31.680610 13621 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.254.130
I20260812 21:03:31.682360 13621 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.254.148:40617
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568607354921-12282-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568607354921-12282-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.130.keytab
--principal=kudu/127.11.254.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.254.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568607354921-12282-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.254.130
--webserver_port=0
--tserver_master_addrs=127.11.254.190:41945
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13621
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.254.130
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568607354921-12282-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:03:31.682670 13621 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:03:31.682904 13621 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:03:31.687976 13630 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:03:31.687987 13631 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:03:31.687987 13634 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:03:31.688251 13621 server_base.cc:1034] running on GCE node
I20260812 21:03:31.688529 13621 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:03:31.688766 13621 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:03:31.689976 13621 hybrid_clock.cc:648] HybridClock initialized: now 1786568611689931 us; error 30 us; skew 500 ppm
Aug 12 21:03:31 dist-test-slave-vpvm krb5kdc[13205](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568611, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:03:31.692673 13621 init.cc:377] Logged in from keytab as kudu/127.11.254.130@KRBTEST.COM (short username kudu)
I20260812 21:03:31.693127 13621 webserver.cc:460] Webserver started at http://127.11.254.130:46787/ using document root <none> and password file <none>
I20260812 21:03:31.693331 13621 fs_manager.cc:359] Metadata directory not provided
I20260812 21:03:31.693389 13621 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:03:31.693516 13621 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:03:31.694640 13621 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568607354921-12282-0/minicluster-data/ts-1/data/instance:
uuid: "68c351d4f45d4b0ba24f725a546ba0bb"
format_stamp: "Formatted at 2026-08-12 21:03:31 on dist-test-slave-vpvm"
server_key: "ad9c6e399bc5fadc2efb0fbd652fa0bb"
server_key_iv: "cdbf4c142ee8d9b53732ba5a86cecbfe"
server_key_version: "encryptionkey@0"
I20260812 21:03:31.694983 13621 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568607354921-12282-0/minicluster-data/ts-1/wal/instance:
uuid: "68c351d4f45d4b0ba24f725a546ba0bb"
format_stamp: "Formatted at 2026-08-12 21:03:31 on dist-test-slave-vpvm"
server_key: "ad9c6e399bc5fadc2efb0fbd652fa0bb"
server_key_iv: "cdbf4c142ee8d9b53732ba5a86cecbfe"
server_key_version: "encryptionkey@0"
I20260812 21:03:31.696327 13621 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 21:03:31.697063 13649 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:03:31.697209 13621 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 21:03:31.697294 13621 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568607354921-12282-0/minicluster-data/ts-1/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568607354921-12282-0/minicluster-data/ts-1/wal
uuid: "68c351d4f45d4b0ba24f725a546ba0bb"
format_stamp: "Formatted at 2026-08-12 21:03:31 on dist-test-slave-vpvm"
server_key: "ad9c6e399bc5fadc2efb0fbd652fa0bb"
server_key_iv: "cdbf4c142ee8d9b53732ba5a86cecbfe"
server_key_version: "encryptionkey@0"
I20260812 21:03:31.697365 13621 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568607354921-12282-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568607354921-12282-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568607354921-12282-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:03:31.715875 13621 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:03:31.718300 13621 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:03:31.718469 13621 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:03:31.718715 13621 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:03:31.719033 13621 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:03:31.719066 13621 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:03:31.719097 13621 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:03:31.719115 13621 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:03:31.724996 13621 rpc_server.cc:307] RPC server started. Bound to: 127.11.254.130:38413
I20260812 21:03:31.725036 13857 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.254.130:38413 every 8 connection(s)
I20260812 21:03:31.725379 13621 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568607354921-12282-0/minicluster-data/ts-1/data/info.pb
Aug 12 21:03:31 dist-test-slave-vpvm krb5kdc[13205](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568611, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.130@KRBTEST.COM for kudu/127.11.254.190@KRBTEST.COM
I20260812 21:03:31.732669 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 13621
I20260812 21:03:31.732746 12282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568607354921-12282-0/minicluster-data/ts-1/wal/instance
I20260812 21:03:31.732896 12282 external_mini_cluster.cc:1428] Setting key 87b64413b1efd0f604d125974f058a91
I20260812 21:03:31.735716 13345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:31.726163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.130:57817 (local address 127.11.254.190:41945)
0812 21:03:31.726211 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0812 21:03:31.726213 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:03:31.726441 (+ 228us) server_negotiation.cc:407] Connection header received
0812 21:03:31.727390 (+ 949us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:03:31.727391 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:03:31.727403 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:03:31.727456 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:03:31.728115 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:31.728680 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:31.729815 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:31.730085 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:31.730159 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:31.732570 (+ 2411us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:03:31.732572 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:03:31.732573 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:03:31.732582 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:03:31.734183 (+ 1601us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:31.734681 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:31.734682 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:31.734682 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:31.734743 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:31.735001 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:31.735002 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:31.735002 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:31.735115 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:03:31.735171 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0812 21:03:31.735513 (+ 342us) server_negotiation.cc:299] Negotiation successful
0812 21:03:31.735618 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 21:03:31.735829 13863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:31.726155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:41945 (local address 127.11.254.130:57817)
0812 21:03:31.726310 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0812 21:03:31.726348 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 21:03:31.727283 (+ 935us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:03:31.727526 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:03:31.727530 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:03:31.727799 (+ 269us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:03:31.728035 (+ 236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:31.728040 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:31.728780 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:31.728781 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:31.729720 (+ 939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:31.729723 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:31.730175 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:31.730177 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:31.730300 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:31.730929 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:03:31.730938 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:03:31.732442 (+ 1504us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:03:31.734282 (+ 1840us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:03:31.734290 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:03:31.734293 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:03:31.734604 (+ 311us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:03:31.734802 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:03:31.734803 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:03:31.734804 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:03:31.734943 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:03:31.735169 (+ 226us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:03:31.735175 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:03:31.735348 (+ 173us) client_negotiation.cc:769] Sending connection context
0812 21:03:31.735468 (+ 120us) client_negotiation.cc:240] Negotiation successful
0812 21:03:31.735641 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":88,"threads_started":1}
I20260812 21:03:31.736373 13860 heartbeater.cc:347] Connected to a master server at 127.11.254.190:41945
I20260812 21:03:31.736477 13860 heartbeater.cc:464] Registering TS with master...
I20260812 21:03:31.736668 13860 heartbeater.cc:511] Master 127.11.254.190:41945 requested a full tablet report, sending...
I20260812 21:03:31.737111 13275 ts_manager.cc:194] Registered new tserver with Master: 68c351d4f45d4b0ba24f725a546ba0bb (127.11.254.130:38413)
I20260812 21:03:31.737594 13275 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.254.130@KRBTEST.COM'} at 127.11.254.130:57817
WARNING: no policy specified for kudu/127.11.254.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.254.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.254.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.131.keytab.
Entry for principal kudu/127.11.254.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.131.keytab.
I20260812 21:03:31.761109 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568607354921-12282-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568607354921-12282-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568607354921-12282-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568607354921-12282-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.254.131:0
--local_ip_for_outbound_sockets=127.11.254.131
--webserver_interface=127.11.254.131
--webserver_port=0
--tserver_master_addrs=127.11.254.190:41945
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.131.keytab
--principal=kudu/127.11.254.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.254.148:40617
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568607354921-12282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568607354921-12282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568607354921-12282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:03:31.846494 13869 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:03:31.846730 13869 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:03:31.846757 13869 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:03:31.847993 13869 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:03:31.848050 13869 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:03:31.848083 13869 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.254.131
I20260812 21:03:31.849669 13869 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.254.148:40617
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568607354921-12282-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568607354921-12282-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.131.keytab
--principal=kudu/127.11.254.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.254.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568607354921-12282-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.254.131
--webserver_port=0
--tserver_master_addrs=127.11.254.190:41945
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13869
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.254.131
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568607354921-12282-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:03:31.849987 13869 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:03:31.850234 13869 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:03:31.855358 13878 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:03:31.855366 13876 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:03:31.855366 13874 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:03:31.855986 13869 server_base.cc:1034] running on GCE node
I20260812 21:03:31.856184 13869 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:03:31.856395 13869 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:03:31.857542 13869 hybrid_clock.cc:648] HybridClock initialized: now 1786568611857528 us; error 29 us; skew 500 ppm
Aug 12 21:03:31 dist-test-slave-vpvm krb5kdc[13205](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568611, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:03:31.859994 13869 init.cc:377] Logged in from keytab as kudu/127.11.254.131@KRBTEST.COM (short username kudu)
I20260812 21:03:31.860394 13869 webserver.cc:460] Webserver started at http://127.11.254.131:44365/ using document root <none> and password file <none>
I20260812 21:03:31.860598 13869 fs_manager.cc:359] Metadata directory not provided
I20260812 21:03:31.860656 13869 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:03:31.860795 13869 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:03:31.861901 13869 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568607354921-12282-0/minicluster-data/ts-2/data/instance:
uuid: "732d42de7f604d8dbaf5825f9230da6a"
format_stamp: "Formatted at 2026-08-12 21:03:31 on dist-test-slave-vpvm"
server_key: "dfae67c9b606f0a2b179726b62e61ae2"
server_key_iv: "aec36a95c3d990d8b339aba245fd3ab4"
server_key_version: "encryptionkey@0"
I20260812 21:03:31.862303 13869 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568607354921-12282-0/minicluster-data/ts-2/wal/instance:
uuid: "732d42de7f604d8dbaf5825f9230da6a"
format_stamp: "Formatted at 2026-08-12 21:03:31 on dist-test-slave-vpvm"
server_key: "dfae67c9b606f0a2b179726b62e61ae2"
server_key_iv: "aec36a95c3d990d8b339aba245fd3ab4"
server_key_version: "encryptionkey@0"
I20260812 21:03:31.863687 13869 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 21:03:31.864488 13890 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:03:31.864668 13869 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 21:03:31.864758 13869 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568607354921-12282-0/minicluster-data/ts-2/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568607354921-12282-0/minicluster-data/ts-2/wal
uuid: "732d42de7f604d8dbaf5825f9230da6a"
format_stamp: "Formatted at 2026-08-12 21:03:31 on dist-test-slave-vpvm"
server_key: "dfae67c9b606f0a2b179726b62e61ae2"
server_key_iv: "aec36a95c3d990d8b339aba245fd3ab4"
server_key_version: "encryptionkey@0"
I20260812 21:03:31.864826 13869 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568607354921-12282-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568607354921-12282-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568607354921-12282-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:03:31.881271 13869 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:03:31.883455 13869 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:03:31.883603 13869 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:03:31.883879 13869 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:03:31.884200 13869 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:03:31.884236 13869 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:03:31.884266 13869 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:03:31.884290 13869 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:03:31.890166 13869 rpc_server.cc:307] RPC server started. Bound to: 127.11.254.131:41647
I20260812 21:03:31.890213 14083 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.254.131:41647 every 8 connection(s)
I20260812 21:03:31.890600 13869 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568607354921-12282-0/minicluster-data/ts-2/data/info.pb
I20260812 21:03:31.897118 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 13869
I20260812 21:03:31.897212 12282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786568607354921-12282-0/minicluster-data/ts-2/wal/instance
I20260812 21:03:31.897380 12282 external_mini_cluster.cc:1428] Setting key f5844de39c2cda889b53584148cc30c8
Aug 12 21:03:31 dist-test-slave-vpvm krb5kdc[13205](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568611, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.131@KRBTEST.COM for kudu/127.11.254.190@KRBTEST.COM
I20260812 21:03:31.900792 13345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:31.891402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.131:32957 (local address 127.11.254.190:41945)
0812 21:03:31.891497 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0812 21:03:31.891499 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:03:31.891771 (+ 272us) server_negotiation.cc:407] Connection header received
0812 21:03:31.892728 (+ 957us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:03:31.892730 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:03:31.892740 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:03:31.892791 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:03:31.893554 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:31.894106 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:31.895364 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:31.895552 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:31.895602 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:31.898309 (+ 2707us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:03:31.898310 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:03:31.898311 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:03:31.898318 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:03:31.899448 (+ 1130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:31.899810 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:31.899810 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:31.899811 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:31.899859 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:31.900061 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:31.900061 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:31.900061 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:31.900212 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:03:31.900282 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 21:03:31.900642 (+ 360us) server_negotiation.cc:299] Negotiation successful
0812 21:03:31.900721 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260812 21:03:31.901016 14090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:31.891367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:41945 (local address 127.11.254.131:32957)
0812 21:03:31.891631 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0812 21:03:31.891668 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0812 21:03:31.892631 (+ 963us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:03:31.892850 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:03:31.892853 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:03:31.893207 (+ 354us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:03:31.893482 (+ 275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:31.893488 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:31.894211 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:31.894213 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:31.895289 (+ 1076us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:31.895292 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:31.895647 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:31.895648 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:31.895811 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:31.896476 (+ 665us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:03:31.896485 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:03:31.898231 (+ 1746us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:03:31.899524 (+ 1293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:03:31.899530 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:03:31.899532 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:03:31.899752 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:03:31.899911 (+ 159us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:03:31.899912 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:03:31.899912 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:03:31.900011 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:03:31.900317 (+ 306us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:03:31.900323 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:03:31.900463 (+ 140us) client_negotiation.cc:769] Sending connection context
0812 21:03:31.900591 (+ 128us) client_negotiation.cc:240] Negotiation successful
0812 21:03:31.900736 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":184,"threads_started":1}
I20260812 21:03:31.901607 14087 heartbeater.cc:347] Connected to a master server at 127.11.254.190:41945
I20260812 21:03:31.901705 14087 heartbeater.cc:464] Registering TS with master...
I20260812 21:03:31.901886 14087 heartbeater.cc:511] Master 127.11.254.190:41945 requested a full tablet report, sending...
I20260812 21:03:31.902321 13275 ts_manager.cc:194] Registered new tserver with Master: 732d42de7f604d8dbaf5825f9230da6a (127.11.254.131:41647)
I20260812 21:03:31.902747 13275 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.254.131@KRBTEST.COM'} at 127.11.254.131:32957
I20260812 21:03:31.909386 12282 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 21:03:31.917520 13345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:31.911526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57174 (local address 127.11.254.190:41945)
0812 21:03:31.911586 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0812 21:03:31.911588 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:03:31.911697 (+ 109us) server_negotiation.cc:407] Connection header received
0812 21:03:31.911766 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:03:31.911767 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:03:31.911778 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:03:31.911826 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:03:31.912385 (+ 559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:31.912894 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:31.913719 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:31.913911 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:31.913984 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:31.915171 (+ 1187us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:03:31.915173 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:03:31.915173 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:03:31.915182 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:03:31.916367 (+ 1185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:31.916709 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:31.916710 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:31.916710 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:31.916771 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:31.916937 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:31.916938 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:31.916938 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:31.917108 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:03:31.917205 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0812 21:03:31.917329 (+ 124us) server_negotiation.cc:299] Negotiation successful
0812 21:03:31.917417 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260812 21:03:31.920713 13275 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57174:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 21:03:31.922781 13275 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 21:03:31.930807 14114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:31.926035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.130:38413 (local address 127.0.0.1:52696)
0812 21:03:31.926269 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0812 21:03:31.926313 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 21:03:31.926384 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:03:31.926684 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:03:31.926687 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:03:31.926689 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:03:31.926765 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:31.926767 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:31.928378 (+ 1611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:31.928379 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:31.929542 (+ 1163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:31.929545 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:31.930477 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:31.930479 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:31.930562 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:31.930579 (+ 17us) client_negotiation.cc:769] Sending connection context
0812 21:03:31.930656 (+ 77us) client_negotiation.cc:240] Negotiation successful
0812 21:03:31.930718 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":201,"mutex_wait_us":5,"thread_start_us":79,"threads_started":1}
I20260812 21:03:31.930931 14113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:31.926035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.131:41647 (local address 127.0.0.1:40386)
0812 21:03:31.926268 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0812 21:03:31.926313 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 21:03:31.926384 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:03:31.926862 (+ 478us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:03:31.926863 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:03:31.926866 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:03:31.926974 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:31.926977 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:31.928550 (+ 1573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:31.928552 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:31.929779 (+ 1227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:31.929782 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:31.930591 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:31.930593 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:31.930699 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:31.930705 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 21:03:31.930769 (+ 64us) client_negotiation.cc:240] Negotiation successful
0812 21:03:31.930816 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"mutex_wait_us":36,"thread_start_us":72,"threads_started":1}
I20260812 21:03:31.931088 14110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:31.926114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.129:42877 (local address 127.0.0.1:49098)
0812 21:03:31.926981 (+ 867us) negotiation.cc:107] Waiting for socket to connect
0812 21:03:31.926991 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 21:03:31.927040 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:03:31.927311 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:03:31.927312 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:03:31.927314 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:03:31.927382 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:31.927384 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:31.928578 (+ 1194us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:31.928579 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:31.929784 (+ 1205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:31.929786 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:31.930876 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:31.930877 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:31.930937 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:31.930941 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 21:03:31.930989 (+ 48us) client_negotiation.cc:240] Negotiation successful
0812 21:03:31.931028 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":854,"mutex_wait_us":52,"thread_start_us":81,"threads_started":1}
I20260812 21:03:31.931160 14120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:31.926229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40386 (local address 127.11.254.131:41647)
0812 21:03:31.926700 (+ 471us) server_negotiation.cc:206] Beginning negotiation
0812 21:03:31.926705 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 21:03:31.926717 (+ 12us) server_negotiation.cc:407] Connection header received
0812 21:03:31.926757 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:03:31.926759 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:03:31.926774 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:03:31.926852 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:03:31.927050 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:31.928284 (+ 1234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:31.929878 (+ 1594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:31.930517 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:31.930589 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:31.930641 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0812 21:03:31.930794 (+ 153us) server_negotiation.cc:299] Negotiation successful
0812 21:03:31.930835 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":434,"thread_start_us":108,"threads_started":1}
I20260812 21:03:31.931272 14121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:31.926245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52696 (local address 127.11.254.130:38413)
0812 21:03:31.926523 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0812 21:03:31.926528 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 21:03:31.926537 (+ 9us) server_negotiation.cc:407] Connection header received
0812 21:03:31.926571 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:03:31.926573 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:03:31.926585 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:03:31.926657 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:03:31.926833 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:31.928262 (+ 1429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:31.929623 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:31.930377 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:31.930459 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:31.930525 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0812 21:03:31.931143 (+ 618us) server_negotiation.cc:299] Negotiation successful
0812 21:03:31.931193 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":246,"thread_start_us":167,"threads_started":1}
I20260812 21:03:31.931416 14126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:31.926477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49098 (local address 127.11.254.129:42877)
0812 21:03:31.926732 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0812 21:03:31.926735 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 21:03:31.927096 (+ 361us) server_negotiation.cc:407] Connection header received
0812 21:03:31.927144 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:03:31.927146 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:03:31.927162 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:03:31.927225 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:03:31.927447 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:31.928317 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:31.929864 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:31.930731 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:31.930793 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:31.930842 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0812 21:03:31.931239 (+ 397us) server_negotiation.cc:299] Negotiation successful
0812 21:03:31.931287 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":188,"thread_start_us":84,"threads_started":1}
I20260812 21:03:31.932077 13975 tablet_service.cc:1467] Processing CreateTablet for tablet 8fa1ff2e8c254ce097c0e4450e49ab88 (DEFAULT_TABLE table=test-table [id=3a721b699bb547f5a31c72c7fb47d5b2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:03:31.932184 13502 tablet_service.cc:1467] Processing CreateTablet for tablet 8fa1ff2e8c254ce097c0e4450e49ab88 (DEFAULT_TABLE table=test-table [id=3a721b699bb547f5a31c72c7fb47d5b2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:03:31.932443 13975 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8fa1ff2e8c254ce097c0e4450e49ab88. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:03:31.932507 13502 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8fa1ff2e8c254ce097c0e4450e49ab88. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:03:31.936475 14127 tablet_bootstrap.cc:492] T 8fa1ff2e8c254ce097c0e4450e49ab88 P 5e25228ef9274b15a1674730bb5ae1b9: Bootstrap starting.
I20260812 21:03:31.936538 14132 tablet_bootstrap.cc:492] T 8fa1ff2e8c254ce097c0e4450e49ab88 P 732d42de7f604d8dbaf5825f9230da6a: Bootstrap starting.
I20260812 21:03:31.937216 13743 tablet_service.cc:1467] Processing CreateTablet for tablet 8fa1ff2e8c254ce097c0e4450e49ab88 (DEFAULT_TABLE table=test-table [id=3a721b699bb547f5a31c72c7fb47d5b2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:03:31.937601 14127 tablet_bootstrap.cc:654] T 8fa1ff2e8c254ce097c0e4450e49ab88 P 5e25228ef9274b15a1674730bb5ae1b9: Neither blocks nor log segments found. Creating new log.
I20260812 21:03:31.937636 13743 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8fa1ff2e8c254ce097c0e4450e49ab88. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:03:31.937609 14132 tablet_bootstrap.cc:654] T 8fa1ff2e8c254ce097c0e4450e49ab88 P 732d42de7f604d8dbaf5825f9230da6a: Neither blocks nor log segments found. Creating new log.
I20260812 21:03:31.938828 14127 log.cc:824] T 8fa1ff2e8c254ce097c0e4450e49ab88 P 5e25228ef9274b15a1674730bb5ae1b9: Log is configured to *not* fsync() on all Append() calls
I20260812 21:03:31.938858 14132 log.cc:824] T 8fa1ff2e8c254ce097c0e4450e49ab88 P 732d42de7f604d8dbaf5825f9230da6a: Log is configured to *not* fsync() on all Append() calls
I20260812 21:03:31.939922 14132 tablet_bootstrap.cc:492] T 8fa1ff2e8c254ce097c0e4450e49ab88 P 732d42de7f604d8dbaf5825f9230da6a: No bootstrap required, opened a new log
I20260812 21:03:31.939961 14127 tablet_bootstrap.cc:492] T 8fa1ff2e8c254ce097c0e4450e49ab88 P 5e25228ef9274b15a1674730bb5ae1b9: No bootstrap required, opened a new log
I20260812 21:03:31.940014 14132 ts_tablet_manager.cc:1397] T 8fa1ff2e8c254ce097c0e4450e49ab88 P 732d42de7f604d8dbaf5825f9230da6a: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260812 21:03:31.940021 14127 ts_tablet_manager.cc:1397] T 8fa1ff2e8c254ce097c0e4450e49ab88 P 5e25228ef9274b15a1674730bb5ae1b9: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20260812 21:03:31.942946 14138 tablet_bootstrap.cc:492] T 8fa1ff2e8c254ce097c0e4450e49ab88 P 68c351d4f45d4b0ba24f725a546ba0bb: Bootstrap starting.
I20260812 21:03:31.943418 14127 raft_consensus.cc:348] T 8fa1ff2e8c254ce097c0e4450e49ab88 P 5e25228ef9274b15a1674730bb5ae1b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "732d42de7f604d8dbaf5825f9230da6a" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 41647 } } peers { permanent_uuid: "68c351d4f45d4b0ba24f725a546ba0bb" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 38413 } } peers { permanent_uuid: "5e25228ef9274b15a1674730bb5ae1b9" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 42877 } }
I20260812 21:03:31.943677 14127 raft_consensus.cc:374] T 8fa1ff2e8c254ce097c0e4450e49ab88 P 5e25228ef9274b15a1674730bb5ae1b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:03:31.943738 14127 raft_consensus.cc:729] T 8fa1ff2e8c254ce097c0e4450e49ab88 P 5e25228ef9274b15a1674730bb5ae1b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e25228ef9274b15a1674730bb5ae1b9, State: Initialized, Role: FOLLOWER
I20260812 21:03:31.943964 14127 consensus_queue.cc:260] T 8fa1ff2e8c254ce097c0e4450e49ab88 P 5e25228ef9274b15a1674730bb5ae1b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "732d42de7f604d8dbaf5825f9230da6a" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 41647 } } peers { permanent_uuid: "68c351d4f45d4b0ba24f725a546ba0bb" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 38413 } } peers { permanent_uuid: "5e25228ef9274b15a1674730bb5ae1b9" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 42877 } }
I20260812 21:03:31.944038 14132 raft_consensus.cc:348] T 8fa1ff2e8c254ce097c0e4450e49ab88 P 732d42de7f604d8dbaf5825f9230da6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "732d42de7f604d8dbaf5825f9230da6a" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 41647 } } peers { permanent_uuid: "68c351d4f45d4b0ba24f725a546ba0bb" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 38413 } } peers { permanent_uuid: "5e25228ef9274b15a1674730bb5ae1b9" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 42877 } }
I20260812 21:03:31.944274 14132 raft_consensus.cc:374] T 8fa1ff2e8c254ce097c0e4450e49ab88 P 732d42de7f604d8dbaf5825f9230da6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:03:31.944327 14132 raft_consensus.cc:729] T 8fa1ff2e8c254ce097c0e4450e49ab88 P 732d42de7f604d8dbaf5825f9230da6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 732d42de7f604d8dbaf5825f9230da6a, State: Initialized, Role: FOLLOWER
I20260812 21:03:31.944366 14138 tablet_bootstrap.cc:654] T 8fa1ff2e8c254ce097c0e4450e49ab88 P 68c351d4f45d4b0ba24f725a546ba0bb: Neither blocks nor log segments found. Creating new log.
I20260812 21:03:31.944475 14132 consensus_queue.cc:260] T 8fa1ff2e8c254ce097c0e4450e49ab88 P 732d42de7f604d8dbaf5825f9230da6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "732d42de7f604d8dbaf5825f9230da6a" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 41647 } } peers { permanent_uuid: "68c351d4f45d4b0ba24f725a546ba0bb" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 38413 } } peers { permanent_uuid: "5e25228ef9274b15a1674730bb5ae1b9" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 42877 } }
I20260812 21:03:31.944543 13610 heartbeater.cc:503] Master 127.11.254.190:41945 was elected leader, sending a full tablet report...
I20260812 21:03:31.944700 14127 ts_tablet_manager.cc:1428] T 8fa1ff2e8c254ce097c0e4450e49ab88 P 5e25228ef9274b15a1674730bb5ae1b9: Time spent starting tablet: real 0.005s user 0.000s sys 0.005s
I20260812 21:03:31.944936 14132 ts_tablet_manager.cc:1428] T 8fa1ff2e8c254ce097c0e4450e49ab88 P 732d42de7f604d8dbaf5825f9230da6a: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260812 21:03:31.945065 14087 heartbeater.cc:503] Master 127.11.254.190:41945 was elected leader, sending a full tablet report...
I20260812 21:03:31.945657 14138 log.cc:824] T 8fa1ff2e8c254ce097c0e4450e49ab88 P 68c351d4f45d4b0ba24f725a546ba0bb: Log is configured to *not* fsync() on all Append() calls
I20260812 21:03:31.946640 14138 tablet_bootstrap.cc:492] T 8fa1ff2e8c254ce097c0e4450e49ab88 P 68c351d4f45d4b0ba24f725a546ba0bb: No bootstrap required, opened a new log
I20260812 21:03:31.946703 14138 ts_tablet_manager.cc:1397] T 8fa1ff2e8c254ce097c0e4450e49ab88 P 68c351d4f45d4b0ba24f725a546ba0bb: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20260812 21:03:31.949816 14138 raft_consensus.cc:348] T 8fa1ff2e8c254ce097c0e4450e49ab88 P 68c351d4f45d4b0ba24f725a546ba0bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "732d42de7f604d8dbaf5825f9230da6a" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 41647 } } peers { permanent_uuid: "68c351d4f45d4b0ba24f725a546ba0bb" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 38413 } } peers { permanent_uuid: "5e25228ef9274b15a1674730bb5ae1b9" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 42877 } }
I20260812 21:03:31.950090 14138 raft_consensus.cc:374] T 8fa1ff2e8c254ce097c0e4450e49ab88 P 68c351d4f45d4b0ba24f725a546ba0bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:03:31.950124 14138 raft_consensus.cc:729] T 8fa1ff2e8c254ce097c0e4450e49ab88 P 68c351d4f45d4b0ba24f725a546ba0bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68c351d4f45d4b0ba24f725a546ba0bb, State: Initialized, Role: FOLLOWER
I20260812 21:03:31.950227 14138 consensus_queue.cc:260] T 8fa1ff2e8c254ce097c0e4450e49ab88 P 68c351d4f45d4b0ba24f725a546ba0bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "732d42de7f604d8dbaf5825f9230da6a" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 41647 } } peers { permanent_uuid: "68c351d4f45d4b0ba24f725a546ba0bb" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 38413 } } peers { permanent_uuid: "5e25228ef9274b15a1674730bb5ae1b9" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 42877 } }
I20260812 21:03:31.950630 14138 ts_tablet_manager.cc:1428] T 8fa1ff2e8c254ce097c0e4450e49ab88 P 68c351d4f45d4b0ba24f725a546ba0bb: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260812 21:03:31.950664 13860 heartbeater.cc:503] Master 127.11.254.190:41945 was elected leader, sending a full tablet report...
I20260812 21:03:31.962884 14140 raft_consensus.cc:482] T 8fa1ff2e8c254ce097c0e4450e49ab88 P 732d42de7f604d8dbaf5825f9230da6a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 21:03:31.963017 14140 raft_consensus.cc:504] T 8fa1ff2e8c254ce097c0e4450e49ab88 P 732d42de7f604d8dbaf5825f9230da6a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "732d42de7f604d8dbaf5825f9230da6a" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 41647 } } peers { permanent_uuid: "68c351d4f45d4b0ba24f725a546ba0bb" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 38413 } } peers { permanent_uuid: "5e25228ef9274b15a1674730bb5ae1b9" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 42877 } }
I20260812 21:03:31.963635 14140 leader_election.cc:288] T 8fa1ff2e8c254ce097c0e4450e49ab88 P 732d42de7f604d8dbaf5825f9230da6a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 68c351d4f45d4b0ba24f725a546ba0bb (127.11.254.130:38413), 5e25228ef9274b15a1674730bb5ae1b9 (127.11.254.129:42877)
I20260812 21:03:31.966953 14126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:31.963916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.131:33497 (local address 127.11.254.129:42877)
0812 21:03:31.963977 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0812 21:03:31.963979 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:03:31.963988 (+ 9us) server_negotiation.cc:407] Connection header received
0812 21:03:31.964037 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:03:31.964038 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:03:31.964046 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:03:31.964092 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:03:31.964356 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:31.964998 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:31.966071 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:31.966543 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:31.966599 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:31.966606 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0812 21:03:31.966858 (+ 252us) server_negotiation.cc:299] Negotiation successful
0812 21:03:31.966887 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260812 21:03:31.966967 14090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:31.963823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.129:42877 (local address 127.11.254.131:33497)
0812 21:03:31.963900 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0812 21:03:31.963921 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 21:03:31.963977 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:03:31.964102 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:03:31.964103 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:03:31.964104 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:03:31.964268 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:31.964270 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:31.965071 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:31.965072 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:31.965995 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:31.965997 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:31.966635 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:31.966636 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:31.966782 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:31.966786 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 21:03:31.966853 (+ 67us) client_negotiation.cc:240] Negotiation successful
0812 21:03:31.966894 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52}
I20260812 21:03:31.967429 13531 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8fa1ff2e8c254ce097c0e4450e49ab88" candidate_uuid: "732d42de7f604d8dbaf5825f9230da6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e25228ef9274b15a1674730bb5ae1b9" is_pre_election: true
I20260812 21:03:31.967633 14121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:31.963914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.131:54789 (local address 127.11.254.130:38413)
0812 21:03:31.963972 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0812 21:03:31.963974 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:03:31.964321 (+ 347us) server_negotiation.cc:407] Connection header received
0812 21:03:31.964353 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:03:31.964355 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:03:31.964364 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:03:31.964458 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:03:31.964711 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:31.965218 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:31.966408 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:31.967093 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:31.967157 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:31.967166 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 21:03:31.967532 (+ 366us) server_negotiation.cc:299] Negotiation successful
0812 21:03:31.967576 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260812 21:03:31.967677 13531 raft_consensus.cc:2454] T 8fa1ff2e8c254ce097c0e4450e49ab88 P 5e25228ef9274b15a1674730bb5ae1b9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 732d42de7f604d8dbaf5825f9230da6a in term 0.
I20260812 21:03:31.967694 14147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:31.963823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.130:38413 (local address 127.11.254.131:54789)
0812 21:03:31.964239 (+ 416us) negotiation.cc:107] Waiting for socket to connect
0812 21:03:31.964248 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 21:03:31.964305 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:03:31.964418 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:03:31.964419 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:03:31.964421 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:03:31.964642 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:31.964644 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:31.965296 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:31.965296 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:31.966312 (+ 1016us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:31.966315 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:31.967347 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:31.967348 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:31.967445 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:31.967450 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 21:03:31.967498 (+ 48us) client_negotiation.cc:240] Negotiation successful
0812 21:03:31.967528 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":395,"mutex_wait_us":118,"thread_start_us":195,"threads_started":1}
I20260812 21:03:31.968000 13893 leader_election.cc:302] T 8fa1ff2e8c254ce097c0e4450e49ab88 P 732d42de7f604d8dbaf5825f9230da6a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e25228ef9274b15a1674730bb5ae1b9, 732d42de7f604d8dbaf5825f9230da6a; no voters:
I20260812 21:03:31.968145 14140 raft_consensus.cc:2781] T 8fa1ff2e8c254ce097c0e4450e49ab88 P 732d42de7f604d8dbaf5825f9230da6a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 21:03:31.968190 14140 raft_consensus.cc:482] T 8fa1ff2e8c254ce097c0e4450e49ab88 P 732d42de7f604d8dbaf5825f9230da6a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 21:03:31.968212 14140 raft_consensus.cc:3037] T 8fa1ff2e8c254ce097c0e4450e49ab88 P 732d42de7f604d8dbaf5825f9230da6a [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:03:31.968137 13776 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8fa1ff2e8c254ce097c0e4450e49ab88" candidate_uuid: "732d42de7f604d8dbaf5825f9230da6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68c351d4f45d4b0ba24f725a546ba0bb" is_pre_election: true
I20260812 21:03:31.968291 13776 raft_consensus.cc:2454] T 8fa1ff2e8c254ce097c0e4450e49ab88 P 68c351d4f45d4b0ba24f725a546ba0bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 732d42de7f604d8dbaf5825f9230da6a in term 0.
I20260812 21:03:31.969460 14140 raft_consensus.cc:504] T 8fa1ff2e8c254ce097c0e4450e49ab88 P 732d42de7f604d8dbaf5825f9230da6a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "732d42de7f604d8dbaf5825f9230da6a" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 41647 } } peers { permanent_uuid: "68c351d4f45d4b0ba24f725a546ba0bb" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 38413 } } peers { permanent_uuid: "5e25228ef9274b15a1674730bb5ae1b9" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 42877 } }
I20260812 21:03:31.969592 14140 leader_election.cc:288] T 8fa1ff2e8c254ce097c0e4450e49ab88 P 732d42de7f604d8dbaf5825f9230da6a [CANDIDATE]: Term 1 election: Requested vote from peers 68c351d4f45d4b0ba24f725a546ba0bb (127.11.254.130:38413), 5e25228ef9274b15a1674730bb5ae1b9 (127.11.254.129:42877)
I20260812 21:03:31.969738 13531 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8fa1ff2e8c254ce097c0e4450e49ab88" candidate_uuid: "732d42de7f604d8dbaf5825f9230da6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e25228ef9274b15a1674730bb5ae1b9"
I20260812 21:03:31.969789 13776 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8fa1ff2e8c254ce097c0e4450e49ab88" candidate_uuid: "732d42de7f604d8dbaf5825f9230da6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68c351d4f45d4b0ba24f725a546ba0bb"
I20260812 21:03:31.969815 13531 raft_consensus.cc:3037] T 8fa1ff2e8c254ce097c0e4450e49ab88 P 5e25228ef9274b15a1674730bb5ae1b9 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:03:31.969869 13776 raft_consensus.cc:3037] T 8fa1ff2e8c254ce097c0e4450e49ab88 P 68c351d4f45d4b0ba24f725a546ba0bb [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:03:31.971110 13776 raft_consensus.cc:2454] T 8fa1ff2e8c254ce097c0e4450e49ab88 P 68c351d4f45d4b0ba24f725a546ba0bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 732d42de7f604d8dbaf5825f9230da6a in term 1.
I20260812 21:03:31.971235 13531 raft_consensus.cc:2454] T 8fa1ff2e8c254ce097c0e4450e49ab88 P 5e25228ef9274b15a1674730bb5ae1b9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 732d42de7f604d8dbaf5825f9230da6a in term 1.
I20260812 21:03:31.971346 13891 leader_election.cc:302] T 8fa1ff2e8c254ce097c0e4450e49ab88 P 732d42de7f604d8dbaf5825f9230da6a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68c351d4f45d4b0ba24f725a546ba0bb, 732d42de7f604d8dbaf5825f9230da6a; no voters:
I20260812 21:03:31.971449 14140 raft_consensus.cc:2781] T 8fa1ff2e8c254ce097c0e4450e49ab88 P 732d42de7f604d8dbaf5825f9230da6a [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:03:31.971724 14140 raft_consensus.cc:686] T 8fa1ff2e8c254ce097c0e4450e49ab88 P 732d42de7f604d8dbaf5825f9230da6a [term 1 LEADER]: Becoming Leader. State: Replica: 732d42de7f604d8dbaf5825f9230da6a, State: Running, Role: LEADER
I20260812 21:03:31.971830 14140 consensus_queue.cc:237] T 8fa1ff2e8c254ce097c0e4450e49ab88 P 732d42de7f604d8dbaf5825f9230da6a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "732d42de7f604d8dbaf5825f9230da6a" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 41647 } } peers { permanent_uuid: "68c351d4f45d4b0ba24f725a546ba0bb" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 38413 } } peers { permanent_uuid: "5e25228ef9274b15a1674730bb5ae1b9" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 42877 } }
I20260812 21:03:31.972766 13275 catalog_manager.cc:5495] T 8fa1ff2e8c254ce097c0e4450e49ab88 P 732d42de7f604d8dbaf5825f9230da6a reported cstate change: term changed from 0 to 1, leader changed from <none> to 732d42de7f604d8dbaf5825f9230da6a (127.11.254.131). New cstate: current_term: 1 leader_uuid: "732d42de7f604d8dbaf5825f9230da6a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "732d42de7f604d8dbaf5825f9230da6a" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 41647 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "68c351d4f45d4b0ba24f725a546ba0bb" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 38413 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e25228ef9274b15a1674730bb5ae1b9" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 42877 } health_report { overall_health: UNKNOWN } } }
W20260812 21:03:31.975986 13862 tablet.cc:2369] T 8fa1ff2e8c254ce097c0e4450e49ab88 P 68c351d4f45d4b0ba24f725a546ba0bb: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 12 21:03:31 dist-test-slave-vpvm krb5kdc[13205](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568611, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.254.129@KRBTEST.COM
I20260812 21:03:31.993125 14126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:31.984985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49100 (local address 127.11.254.129:42877)
0812 21:03:31.985047 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0812 21:03:31.985049 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:03:31.985189 (+ 140us) server_negotiation.cc:407] Connection header received
0812 21:03:31.985258 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:03:31.985259 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:03:31.985270 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:03:31.985316 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:03:31.986032 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:31.986523 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:31.987322 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:31.987515 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:31.987580 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:31.990142 (+ 2562us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:03:31.990154 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:03:31.990154 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:03:31.990162 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:03:31.991982 (+ 1820us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:31.992326 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:31.992330 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:31.992330 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:31.992386 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:31.992574 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:31.992575 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:31.992575 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:31.992721 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:03:31.992866 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0812 21:03:31.992946 (+ 80us) server_negotiation.cc:299] Negotiation successful
0812 21:03:31.993030 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
Aug 12 21:03:32 dist-test-slave-vpvm krb5kdc[13205](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568612, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 21:03:32 dist-test-slave-vpvm krb5kdc[13205](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568612, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.11.254.129@KRBTEST.COM
I20260812 21:03:32.019726 14126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:32.012255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49110 (local address 127.11.254.129:42877)
0812 21:03:32.012336 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0812 21:03:32.012338 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:03:32.012373 (+ 35us) server_negotiation.cc:407] Connection header received
0812 21:03:32.012439 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:03:32.012440 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:03:32.012450 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:03:32.012500 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:03:32.013107 (+ 607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:32.013601 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:32.014364 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:32.014598 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:32.014657 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:32.017070 (+ 2413us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:03:32.017072 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:03:32.017072 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:03:32.017081 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:03:32.018498 (+ 1417us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:32.018825 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:32.018825 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:32.018826 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:32.018883 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:32.019119 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:32.019120 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:32.019121 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:32.019240 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:03:32.019326 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0812 21:03:32.019518 (+ 192us) server_negotiation.cc:299] Negotiation successful
0812 21:03:32.019596 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
Aug 12 21:03:32 dist-test-slave-vpvm krb5kdc[13205](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568612, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 21:03:32 dist-test-slave-vpvm krb5kdc[13205](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568612, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.254.129@KRBTEST.COM
I20260812 21:03:32.044000 14126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:32.037248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49112 (local address 127.11.254.129:42877)
0812 21:03:32.037310 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0812 21:03:32.037312 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:03:32.037396 (+ 84us) server_negotiation.cc:407] Connection header received
0812 21:03:32.037442 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:03:32.037443 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:03:32.037452 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:03:32.037502 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:03:32.038121 (+ 619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:32.038594 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:32.039321 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:32.039493 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:32.039535 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:32.041863 (+ 2328us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:03:32.041865 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:03:32.041865 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:03:32.041873 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:03:32.043099 (+ 1226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:32.043411 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:32.043412 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:32.043412 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:32.043459 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:32.043588 (+ 129us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:32.043588 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:32.043589 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:32.043711 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:03:32.043785 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0812 21:03:32.043868 (+ 83us) server_negotiation.cc:299] Negotiation successful
0812 21:03:32.043928 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
W20260812 21:03:32.044682 13469 tablet_service.cc:2908] Rejecting scan request for tablet 8fa1ff2e8c254ce097c0e4450e49ab88: Uninitialized: clean time has not yet been initialized
I20260812 21:03:32.045594 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 13386
I20260812 21:03:32.063678 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 13621
I20260812 21:03:32.072580 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 13869
I20260812 21:03:32.085598 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 13224
2026-08-12T21:03:32Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2215 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568607354921-12282-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 21:03:32 dist-test-slave-vpvm krb5kdc[14170](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 21:03:32 dist-test-slave-vpvm krb5kdc[14170](info): set up 2 sockets
Aug 12 21:03:32 dist-test-slave-vpvm krb5kdc[14170](info): commencing operation
krb5kdc: starting...
W20260812 21:03:33.270640 12282 mini_kdc.cc:121] Time spent starting KDC: real 1.155s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568607354921-12282-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568607354921-12282-0/krb5kdc/test-user.keytab.
Aug 12 21:03:33 dist-test-slave-vpvm krb5kdc[14170](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568613, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T21:03:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T21:03:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.254.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.254.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.254.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.190.keytab.
Entry for principal kudu/127.11.254.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.190.keytab.
I20260812 21:03:33.396338 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568607354921-12282-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568607354921-12282-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568607354921-12282-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.254.190:42795
--webserver_interface=127.11.254.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.190.keytab
--principal=kudu/127.11.254.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.254.148:34211
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.254.190:42795
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568607354921-12282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568607354921-12282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568607354921-12282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:03:33.479126 14193 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:03:33.479352 14193 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:03:33.479374 14193 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:03:33.480480 14193 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 21:03:33.480516 14193 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260812 21:03:33.480530 14193 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:03:33.480541 14193 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:03:33.480559 14193 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 21:03:33.480571 14193 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 21:03:33.482059 14193 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.254.148:34211
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568607354921-12282-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568607354921-12282-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.254.190:42795
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.190.keytab
--principal=kudu/127.11.254.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.254.190:42795
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.254.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14193
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568607354921-12282-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:03:33.482265 14193 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:03:33.482456 14193 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:03:33.487311 14202 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:03:33.487320 14208 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:03:33.487435 14193 server_base.cc:1034] running on GCE node
W20260812 21:03:33.487521 14203 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:03:33.487874 14193 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:03:33.488166 14193 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:03:33.489306 14193 hybrid_clock.cc:648] HybridClock initialized: now 1786568613489288 us; error 31 us; skew 500 ppm
Aug 12 21:03:33 dist-test-slave-vpvm krb5kdc[14170](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568613, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:03:33.491832 14193 init.cc:377] Logged in from keytab as kudu/127.11.254.190@KRBTEST.COM (short username kudu)
I20260812 21:03:33.492252 14193 webserver.cc:460] Webserver started at http://127.11.254.190:44075/ using document root <none> and password file <none>
I20260812 21:03:33.492431 14193 fs_manager.cc:359] Metadata directory not provided
I20260812 21:03:33.492470 14193 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:03:33.492563 14193 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:03:33.493587 14193 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568607354921-12282-0/minicluster-data/master-0/data/instance:
uuid: "f45b7349c97e46c59225cf0ba1c645e0"
format_stamp: "Formatted at 2026-08-12 21:03:33 on dist-test-slave-vpvm"
server_key: "a8fa3e2b8b2b1992916ffc1a1699d9bb"
server_key_iv: "15146ec72a0d20fad52a2fc7f76246b5"
server_key_version: "encryptionkey@0"
I20260812 21:03:33.493894 14193 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568607354921-12282-0/minicluster-data/master-0/wal/instance:
uuid: "f45b7349c97e46c59225cf0ba1c645e0"
format_stamp: "Formatted at 2026-08-12 21:03:33 on dist-test-slave-vpvm"
server_key: "a8fa3e2b8b2b1992916ffc1a1699d9bb"
server_key_iv: "15146ec72a0d20fad52a2fc7f76246b5"
server_key_version: "encryptionkey@0"
I20260812 21:03:33.495177 14193 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 21:03:33.495822 14216 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:03:33.495980 14193 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 21:03:33.496065 14193 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568607354921-12282-0/minicluster-data/master-0/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568607354921-12282-0/minicluster-data/master-0/wal
uuid: "f45b7349c97e46c59225cf0ba1c645e0"
format_stamp: "Formatted at 2026-08-12 21:03:33 on dist-test-slave-vpvm"
server_key: "a8fa3e2b8b2b1992916ffc1a1699d9bb"
server_key_iv: "15146ec72a0d20fad52a2fc7f76246b5"
server_key_version: "encryptionkey@0"
I20260812 21:03:33.496125 14193 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568607354921-12282-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568607354921-12282-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568607354921-12282-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:03:33.504491 14193 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:03:33.506522 14193 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:03:33.506651 14193 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:03:33.512822 14193 rpc_server.cc:307] RPC server started. Bound to: 127.11.254.190:42795
I20260812 21:03:33.512956 14325 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.254.190:42795 every 8 connection(s)
I20260812 21:03:33.513482 14193 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
I20260812 21:03:33.514755 14326 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:03:33.519254 14326 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f45b7349c97e46c59225cf0ba1c645e0: Bootstrap starting.
I20260812 21:03:33.520251 14326 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f45b7349c97e46c59225cf0ba1c645e0: Neither blocks nor log segments found. Creating new log.
I20260812 21:03:33.520678 14326 log.cc:824] T 00000000000000000000000000000000 P f45b7349c97e46c59225cf0ba1c645e0: Log is configured to *not* fsync() on all Append() calls
I20260812 21:03:33.522089 14326 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f45b7349c97e46c59225cf0ba1c645e0: No bootstrap required, opened a new log
I20260812 21:03:33.522164 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 14193
I20260812 21:03:33.522328 12282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568607354921-12282-0/minicluster-data/master-0/wal/instance
I20260812 21:03:33.522678 12282 external_mini_cluster.cc:1428] Setting key 82d01401a10133b8bb45d6303cb3f391
I20260812 21:03:33.528038 14326 raft_consensus.cc:348] T 00000000000000000000000000000000 P f45b7349c97e46c59225cf0ba1c645e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f45b7349c97e46c59225cf0ba1c645e0" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 42795 } }
I20260812 21:03:33.528393 14326 raft_consensus.cc:374] T 00000000000000000000000000000000 P f45b7349c97e46c59225cf0ba1c645e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:03:33.528515 14326 raft_consensus.cc:729] T 00000000000000000000000000000000 P f45b7349c97e46c59225cf0ba1c645e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f45b7349c97e46c59225cf0ba1c645e0, State: Initialized, Role: FOLLOWER
I20260812 21:03:33.528841 14326 consensus_queue.cc:260] T 00000000000000000000000000000000 P f45b7349c97e46c59225cf0ba1c645e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f45b7349c97e46c59225cf0ba1c645e0" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 42795 } }
I20260812 21:03:33.529055 14326 raft_consensus.cc:388] T 00000000000000000000000000000000 P f45b7349c97e46c59225cf0ba1c645e0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 21:03:33.529147 14326 raft_consensus.cc:482] T 00000000000000000000000000000000 P f45b7349c97e46c59225cf0ba1c645e0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 21:03:33.529284 14326 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f45b7349c97e46c59225cf0ba1c645e0 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:03:33.530848 14326 raft_consensus.cc:504] T 00000000000000000000000000000000 P f45b7349c97e46c59225cf0ba1c645e0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f45b7349c97e46c59225cf0ba1c645e0" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 42795 } }
I20260812 21:03:33.531069 14326 leader_election.cc:302] T 00000000000000000000000000000000 P f45b7349c97e46c59225cf0ba1c645e0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f45b7349c97e46c59225cf0ba1c645e0; no voters:
I20260812 21:03:33.531473 14326 leader_election.cc:288] T 00000000000000000000000000000000 P f45b7349c97e46c59225cf0ba1c645e0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 21:03:33.531572 14333 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f45b7349c97e46c59225cf0ba1c645e0 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:03:33.531909 14333 raft_consensus.cc:686] T 00000000000000000000000000000000 P f45b7349c97e46c59225cf0ba1c645e0 [term 1 LEADER]: Becoming Leader. State: Replica: f45b7349c97e46c59225cf0ba1c645e0, State: Running, Role: LEADER
I20260812 21:03:33.532156 14333 consensus_queue.cc:237] T 00000000000000000000000000000000 P f45b7349c97e46c59225cf0ba1c645e0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f45b7349c97e46c59225cf0ba1c645e0" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 42795 } }
I20260812 21:03:33.532193 14326 sys_catalog.cc:564] T 00000000000000000000000000000000 P f45b7349c97e46c59225cf0ba1c645e0 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 21:03:33.532805 14336 sys_catalog.cc:455] T 00000000000000000000000000000000 P f45b7349c97e46c59225cf0ba1c645e0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f45b7349c97e46c59225cf0ba1c645e0. Latest consensus state: current_term: 1 leader_uuid: "f45b7349c97e46c59225cf0ba1c645e0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f45b7349c97e46c59225cf0ba1c645e0" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 42795 } } }
I20260812 21:03:33.532922 14336 sys_catalog.cc:458] T 00000000000000000000000000000000 P f45b7349c97e46c59225cf0ba1c645e0 [sys.catalog]: This master's current role is: LEADER
I20260812 21:03:33.533424 14338 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 21:03:33.534469 14338 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 21:03:33.534855 14334 sys_catalog.cc:455] T 00000000000000000000000000000000 P f45b7349c97e46c59225cf0ba1c645e0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f45b7349c97e46c59225cf0ba1c645e0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f45b7349c97e46c59225cf0ba1c645e0" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 42795 } } }
I20260812 21:03:33.535019 14334 sys_catalog.cc:458] T 00000000000000000000000000000000 P f45b7349c97e46c59225cf0ba1c645e0 [sys.catalog]: This master's current role is: LEADER
Aug 12 21:03:33 dist-test-slave-vpvm krb5kdc[14170](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568613, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.254.190@KRBTEST.COM
I20260812 21:03:33.538278 14338 catalog_manager.cc:1347] Generated new cluster ID: be1a4849b660405f935aa31a98eafeab
I20260812 21:03:33.538393 14338 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 21:03:33.542590 14332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:33.523577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40130 (local address 127.11.254.190:42795)
0812 21:03:33.523889 (+ 312us) server_negotiation.cc:206] Beginning negotiation
0812 21:03:33.523906 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0812 21:03:33.523971 (+ 65us) server_negotiation.cc:407] Connection header received
0812 21:03:33.524708 (+ 737us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:03:33.524719 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:03:33.524834 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:03:33.525003 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:03:33.526793 (+ 1790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:33.528278 (+ 1485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:33.530120 (+ 1842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:33.530886 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:33.531075 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:33.536809 (+ 5734us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:03:33.536822 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:03:33.536823 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:03:33.536849 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:03:33.539856 (+ 3007us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:33.540533 (+ 677us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:33.540537 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:33.540538 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:33.540706 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:33.540984 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:33.540985 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:33.540986 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:33.541462 (+ 476us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:03:33.541722 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0812 21:03:33.541943 (+ 221us) server_negotiation.cc:299] Negotiation successful
0812 21:03:33.542270 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":245,"thread_start_us":177,"threads_started":1}
I20260812 21:03:33.549063 14338 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 21:03:33.550143 14338 catalog_manager.cc:1504] Loading token signing keys...
I20260812 21:03:33.572057 14338 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f45b7349c97e46c59225cf0ba1c645e0: Generated new TSK 0
I20260812 21:03:33.572400 14338 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.254.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.254.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.254.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.129.keytab.
Entry for principal kudu/127.11.254.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.129.keytab.
I20260812 21:03:33.622363 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568607354921-12282-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568607354921-12282-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568607354921-12282-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568607354921-12282-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.254.129:0
--local_ip_for_outbound_sockets=127.11.254.129
--webserver_interface=127.11.254.129
--webserver_port=0
--tserver_master_addrs=127.11.254.190:42795
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.129.keytab
--principal=kudu/127.11.254.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.254.148:34211
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568607354921-12282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568607354921-12282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568607354921-12282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:03:33.806928 14371 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:03:33.807299 14371 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:03:33.807344 14371 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:03:33.809213 14371 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:03:33.809301 14371 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:03:33.809337 14371 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 21:03:33.809378 14371 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.254.129
I20260812 21:03:33.811916 14371 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.254.148:34211
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568607354921-12282-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568607354921-12282-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.129.keytab
--principal=kudu/127.11.254.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.254.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568607354921-12282-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.254.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.11.254.190:42795
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14371
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.254.129
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568607354921-12282-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:03:33.812333 14371 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:03:33.812752 14371 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:03:33.820940 14388 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:03:33.820950 14398 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:03:33.820953 14391 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:03:33.821688 14371 server_base.cc:1034] running on GCE node
I20260812 21:03:33.822083 14371 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:03:33.822489 14371 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:03:33.823715 14371 hybrid_clock.cc:648] HybridClock initialized: now 1786568613823697 us; error 46 us; skew 500 ppm
Aug 12 21:03:33 dist-test-slave-vpvm krb5kdc[14170](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568613, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:03:33.828342 14371 init.cc:377] Logged in from keytab as kudu/127.11.254.129@KRBTEST.COM (short username kudu)
I20260812 21:03:33.829306 14371 webserver.cc:460] Webserver started at http://127.11.254.129:38567/ using document root <none> and password file <none>
I20260812 21:03:33.829627 14371 fs_manager.cc:359] Metadata directory not provided
I20260812 21:03:33.829707 14371 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:03:33.829892 14371 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:03:33.832146 14371 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568607354921-12282-0/minicluster-data/ts-0/data/instance:
uuid: "e619668bf91d4868bc386cbf6f07c3a2"
format_stamp: "Formatted at 2026-08-12 21:03:33 on dist-test-slave-vpvm"
server_key: "f0266a56e7db8215e2ac4035846534ed"
server_key_iv: "d2abeaf9304cd473cc951ea452d7bac4"
server_key_version: "encryptionkey@0"
I20260812 21:03:33.832719 14371 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568607354921-12282-0/minicluster-data/ts-0/wal/instance:
uuid: "e619668bf91d4868bc386cbf6f07c3a2"
format_stamp: "Formatted at 2026-08-12 21:03:33 on dist-test-slave-vpvm"
server_key: "f0266a56e7db8215e2ac4035846534ed"
server_key_iv: "d2abeaf9304cd473cc951ea452d7bac4"
server_key_version: "encryptionkey@0"
I20260812 21:03:33.835247 14371 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.005s sys 0.000s
I20260812 21:03:33.836483 14411 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:03:33.836803 14371 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 21:03:33.836957 14371 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568607354921-12282-0/minicluster-data/ts-0/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568607354921-12282-0/minicluster-data/ts-0/wal
uuid: "e619668bf91d4868bc386cbf6f07c3a2"
format_stamp: "Formatted at 2026-08-12 21:03:33 on dist-test-slave-vpvm"
server_key: "f0266a56e7db8215e2ac4035846534ed"
server_key_iv: "d2abeaf9304cd473cc951ea452d7bac4"
server_key_version: "encryptionkey@0"
I20260812 21:03:33.837105 14371 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568607354921-12282-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568607354921-12282-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568607354921-12282-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:03:33.848409 14371 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:03:33.852082 14371 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:03:33.852334 14371 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:03:33.853181 14371 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:03:33.853247 14371 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:03:33.853287 14371 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:03:33.853335 14371 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 21:03:33 dist-test-slave-vpvm krb5kdc[14170](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568613, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.129@KRBTEST.COM for kudu/127.11.254.190@KRBTEST.COM
I20260812 21:03:33.866550 14371 rpc_server.cc:307] RPC server started. Bound to: 127.11.254.129:42771
I20260812 21:03:33.866822 14621 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.254.129:42771 every 8 connection(s)
I20260812 21:03:33.867340 14371 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568607354921-12282-0/minicluster-data/ts-0/data/info.pb
I20260812 21:03:33.871543 14332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:33.854045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.129:49925 (local address 127.11.254.190:42795)
0812 21:03:33.854158 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0812 21:03:33.854161 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 21:03:33.854506 (+ 345us) server_negotiation.cc:407] Connection header received
0812 21:03:33.855300 (+ 794us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:03:33.855301 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:03:33.855326 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:03:33.855491 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:03:33.856506 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:33.857721 (+ 1215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:33.860456 (+ 2735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:33.860991 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:33.861145 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:33.867116 (+ 5971us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:03:33.867118 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:03:33.867118 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:03:33.867128 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:03:33.869390 (+ 2262us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:33.870119 (+ 729us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:33.870120 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:33.870120 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:33.870216 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:33.870570 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:33.870571 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:33.870571 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:33.870779 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:03:33.870867 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0812 21:03:33.871301 (+ 434us) server_negotiation.cc:299] Negotiation successful
0812 21:03:33.871444 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260812 21:03:33.871872 14426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:33.854014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:42795 (local address 127.11.254.129:49925)
0812 21:03:33.854306 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0812 21:03:33.854386 (+ 80us) client_negotiation.cc:174] Beginning negotiation
0812 21:03:33.855137 (+ 751us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:03:33.855586 (+ 449us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:03:33.855589 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:03:33.856007 (+ 418us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:03:33.856342 (+ 335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:33.856347 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:33.857863 (+ 1516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:33.857864 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:33.860257 (+ 2393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:33.860262 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:33.861159 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:33.861161 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:33.861458 (+ 297us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:33.862605 (+ 1147us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:03:33.862619 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:03:33.866895 (+ 4276us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:03:33.869499 (+ 2604us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:03:33.869506 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:03:33.869508 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:03:33.869999 (+ 491us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:03:33.870304 (+ 305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:03:33.870305 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:03:33.870305 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:03:33.870489 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:03:33.870853 (+ 364us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:03:33.870857 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:03:33.871083 (+ 226us) client_negotiation.cc:769] Sending connection context
0812 21:03:33.871247 (+ 164us) client_negotiation.cc:240] Negotiation successful
0812 21:03:33.871500 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":149,"threads_started":1}
I20260812 21:03:33.872677 14622 heartbeater.cc:347] Connected to a master server at 127.11.254.190:42795
I20260812 21:03:33.872820 14622 heartbeater.cc:464] Registering TS with master...
I20260812 21:03:33.873143 14622 heartbeater.cc:511] Master 127.11.254.190:42795 requested a full tablet report, sending...
I20260812 21:03:33.874038 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 14371
I20260812 21:03:33.874168 12282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568607354921-12282-0/minicluster-data/ts-0/wal/instance
I20260812 21:03:33.874171 14246 ts_manager.cc:194] Registered new tserver with Master: e619668bf91d4868bc386cbf6f07c3a2 (127.11.254.129:42771)
I20260812 21:03:33.874544 12282 external_mini_cluster.cc:1428] Setting key da0c407ccdf1a83fc8866a1fae4f1ec7
I20260812 21:03:33.875586 14246 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.254.129@KRBTEST.COM'} at 127.11.254.129:49925
I20260812 21:03:33.884608 14633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:33.874549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:42795 (local address 127.11.254.129:44219)
0812 21:03:33.874811 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0812 21:03:33.874831 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 21:03:33.874913 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:03:33.875087 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:03:33.875096 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:03:33.875472 (+ 376us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:03:33.875590 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:33.875593 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:33.876878 (+ 1285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:33.876879 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:33.877654 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:33.877665 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:33.878404 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:33.878405 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:33.878552 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:33.879248 (+ 696us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:03:33.879254 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:03:33.879996 (+ 742us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:03:33.882986 (+ 2990us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:03:33.882987 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:03:33.882987 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:03:33.883305 (+ 318us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:03:33.883595 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:03:33.883596 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:03:33.883596 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:03:33.883697 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:03:33.884161 (+ 464us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:03:33.884162 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:03:33.884269 (+ 107us) client_negotiation.cc:769] Sending connection context
0812 21:03:33.884351 (+ 82us) client_negotiation.cc:240] Negotiation successful
0812 21:03:33.884486 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":138,"threads_started":1}
I20260812 21:03:33.884703 14332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:33.874856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.129:44219 (local address 127.11.254.190:42795)
0812 21:03:33.874928 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0812 21:03:33.874930 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:03:33.874973 (+ 43us) server_negotiation.cc:407] Connection header received
0812 21:03:33.875004 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:03:33.875005 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:03:33.875015 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:03:33.875085 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:03:33.875727 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:33.876711 (+ 984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:33.877782 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:33.878309 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:33.878396 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:33.880140 (+ 1744us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:03:33.880141 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:03:33.880142 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:03:33.880150 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:03:33.882774 (+ 2624us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:33.883403 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:33.883405 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:33.883405 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:33.883501 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:33.883776 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:33.883777 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:33.883777 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:33.884035 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:03:33.884172 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0812 21:03:33.884422 (+ 250us) server_negotiation.cc:299] Negotiation successful
0812 21:03:33.884564 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
WARNING: no policy specified for kudu/127.11.254.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.254.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.254.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.130.keytab.
Entry for principal kudu/127.11.254.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.130.keytab.
I20260812 21:03:33.913923 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568607354921-12282-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568607354921-12282-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568607354921-12282-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568607354921-12282-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.254.130:0
--local_ip_for_outbound_sockets=127.11.254.130
--webserver_interface=127.11.254.130
--webserver_port=0
--tserver_master_addrs=127.11.254.190:42795
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.130.keytab
--principal=kudu/127.11.254.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.254.148:34211
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568607354921-12282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568607354921-12282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568607354921-12282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:03:34.057085 14646 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:03:34.057507 14646 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:03:34.057561 14646 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:03:34.059538 14646 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:03:34.059623 14646 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:03:34.059665 14646 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 21:03:34.059710 14646 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.254.130
I20260812 21:03:34.062191 14646 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.254.148:34211
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568607354921-12282-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568607354921-12282-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.130.keytab
--principal=kudu/127.11.254.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.254.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568607354921-12282-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.254.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.11.254.190:42795
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14646
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.254.130
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568607354921-12282-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:03:34.062669 14646 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:03:34.063033 14646 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:03:34.071223 14653 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:03:34.071215 14659 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:03:34.071218 14655 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:03:34.071605 14646 server_base.cc:1034] running on GCE node
I20260812 21:03:34.072000 14646 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:03:34.072443 14646 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:03:34.073695 14646 hybrid_clock.cc:648] HybridClock initialized: now 1786568614073679 us; error 65 us; skew 500 ppm
Aug 12 21:03:34 dist-test-slave-vpvm krb5kdc[14170](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568614, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:03:34.078156 14646 init.cc:377] Logged in from keytab as kudu/127.11.254.130@KRBTEST.COM (short username kudu)
I20260812 21:03:34.078977 14646 webserver.cc:460] Webserver started at http://127.11.254.130:42713/ using document root <none> and password file <none>
I20260812 21:03:34.079355 14646 fs_manager.cc:359] Metadata directory not provided
I20260812 21:03:34.079450 14646 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:03:34.079710 14646 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:03:34.081516 14646 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568607354921-12282-0/minicluster-data/ts-1/data/instance:
uuid: "ef41ad9d59be4951885e7143db439d06"
format_stamp: "Formatted at 2026-08-12 21:03:34 on dist-test-slave-vpvm"
server_key: "aab8f81e1568c1429322fd60c20e3db8"
server_key_iv: "c999b14bed215bae613b05bfb300da4c"
server_key_version: "encryptionkey@0"
I20260812 21:03:34.082281 14646 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568607354921-12282-0/minicluster-data/ts-1/wal/instance:
uuid: "ef41ad9d59be4951885e7143db439d06"
format_stamp: "Formatted at 2026-08-12 21:03:34 on dist-test-slave-vpvm"
server_key: "aab8f81e1568c1429322fd60c20e3db8"
server_key_iv: "c999b14bed215bae613b05bfb300da4c"
server_key_version: "encryptionkey@0"
I20260812 21:03:34.084785 14646 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.001s
I20260812 21:03:34.086107 14666 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:03:34.086362 14646 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20260812 21:03:34.086565 14646 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568607354921-12282-0/minicluster-data/ts-1/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568607354921-12282-0/minicluster-data/ts-1/wal
uuid: "ef41ad9d59be4951885e7143db439d06"
format_stamp: "Formatted at 2026-08-12 21:03:34 on dist-test-slave-vpvm"
server_key: "aab8f81e1568c1429322fd60c20e3db8"
server_key_iv: "c999b14bed215bae613b05bfb300da4c"
server_key_version: "encryptionkey@0"
I20260812 21:03:34.086680 14646 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568607354921-12282-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568607354921-12282-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568607354921-12282-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:03:34.099359 14646 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:03:34.102854 14646 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:03:34.103104 14646 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:03:34.103960 14646 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:03:34.104053 14646 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:03:34.104120 14646 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:03:34.104167 14646 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 21:03:34 dist-test-slave-vpvm krb5kdc[14170](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568614, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.130@KRBTEST.COM for kudu/127.11.254.190@KRBTEST.COM
I20260812 21:03:34.116823 14646 rpc_server.cc:307] RPC server started. Bound to: 127.11.254.130:45471
I20260812 21:03:34.116907 14863 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.254.130:45471 every 8 connection(s)
I20260812 21:03:34.117631 14646 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568607354921-12282-0/minicluster-data/ts-1/data/info.pb
I20260812 21:03:34.119789 14332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:34.104691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.130:52587 (local address 127.11.254.190:42795)
0812 21:03:34.104813 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0812 21:03:34.104816 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 21:03:34.105106 (+ 290us) server_negotiation.cc:407] Connection header received
0812 21:03:34.105833 (+ 727us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:03:34.105834 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:03:34.105857 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:03:34.106005 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:03:34.107150 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:34.107851 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:34.109468 (+ 1617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:34.109838 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:34.109939 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:34.114305 (+ 4366us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:03:34.114307 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:03:34.114308 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:03:34.114325 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:03:34.117304 (+ 2979us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:34.118177 (+ 873us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:34.118178 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:34.118179 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:34.118314 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:34.118706 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:34.118707 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:34.118707 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:34.118918 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:03:34.119019 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0812 21:03:34.119460 (+ 441us) server_negotiation.cc:299] Negotiation successful
0812 21:03:34.119628 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260812 21:03:34.120132 14677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:34.104611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:42795 (local address 127.11.254.130:52587)
0812 21:03:34.104898 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0812 21:03:34.104971 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0812 21:03:34.105690 (+ 719us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:03:34.106133 (+ 443us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:03:34.106139 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:03:34.106639 (+ 500us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:03:34.106976 (+ 337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:34.106995 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:34.107994 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:34.107995 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:34.109332 (+ 1337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:34.109335 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:34.110004 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:34.110006 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:34.110250 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:34.111092 (+ 842us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:03:34.111115 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:03:34.114162 (+ 3047us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:03:34.117471 (+ 3309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:03:34.117482 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:03:34.117485 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:03:34.117929 (+ 444us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:03:34.118419 (+ 490us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:03:34.118420 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:03:34.118420 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:03:34.118583 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:03:34.119011 (+ 428us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:03:34.119019 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:03:34.119231 (+ 212us) client_negotiation.cc:769] Sending connection context
0812 21:03:34.119412 (+ 181us) client_negotiation.cc:240] Negotiation successful
0812 21:03:34.119742 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":151,"threads_started":1}
I20260812 21:03:34.120893 14864 heartbeater.cc:347] Connected to a master server at 127.11.254.190:42795
I20260812 21:03:34.121038 14864 heartbeater.cc:464] Registering TS with master...
I20260812 21:03:34.121402 14864 heartbeater.cc:511] Master 127.11.254.190:42795 requested a full tablet report, sending...
I20260812 21:03:34.122115 14246 ts_manager.cc:194] Registered new tserver with Master: ef41ad9d59be4951885e7143db439d06 (127.11.254.130:45471)
I20260812 21:03:34.122758 14246 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.254.130@KRBTEST.COM'} at 127.11.254.130:52587
I20260812 21:03:34.123634 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 14646
I20260812 21:03:34.123783 12282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568607354921-12282-0/minicluster-data/ts-1/wal/instance
I20260812 21:03:34.124133 12282 external_mini_cluster.cc:1428] Setting key 8092d2343f42eb68b908d74ae8241792
I20260812 21:03:34.131031 14332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:34.122771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.130:54409 (local address 127.11.254.190:42795)
0812 21:03:34.122826 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0812 21:03:34.122829 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 21:03:34.123013 (+ 184us) server_negotiation.cc:407] Connection header received
0812 21:03:34.123124 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:03:34.123124 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:03:34.123145 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:03:34.123211 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:03:34.123692 (+ 481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:34.124611 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:34.125594 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:34.125863 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:34.125944 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:34.127297 (+ 1353us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:03:34.127306 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:03:34.127306 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:03:34.127315 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:03:34.129269 (+ 1954us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:34.129721 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:34.129722 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:34.129723 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:34.129811 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:34.130139 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:34.130140 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:34.130140 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:34.130365 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:03:34.130466 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0812 21:03:34.130713 (+ 247us) server_negotiation.cc:299] Negotiation successful
0812 21:03:34.130862 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 21:03:34.131055 14877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:34.122700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:42795 (local address 127.11.254.130:54409)
0812 21:03:34.122921 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0812 21:03:34.122938 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 21:03:34.123027 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:03:34.123224 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:03:34.123225 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:03:34.123491 (+ 266us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:03:34.123587 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:34.123589 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:34.124727 (+ 1138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:34.124728 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:34.125503 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:34.125506 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:34.125939 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:34.125945 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:34.126100 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:34.126710 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:03:34.126721 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:03:34.127200 (+ 479us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:03:34.129370 (+ 2170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:03:34.129371 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:03:34.129372 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:03:34.129628 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:03:34.129899 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:03:34.129900 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:03:34.129900 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:03:34.130047 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:03:34.130462 (+ 415us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:03:34.130463 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:03:34.130550 (+ 87us) client_negotiation.cc:769] Sending connection context
0812 21:03:34.130666 (+ 116us) client_negotiation.cc:240] Negotiation successful
0812 21:03:34.130862 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":106,"threads_started":1}
WARNING: no policy specified for kudu/127.11.254.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.254.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.254.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.131.keytab.
Entry for principal kudu/127.11.254.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.131.keytab.
I20260812 21:03:34.163200 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568607354921-12282-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568607354921-12282-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568607354921-12282-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568607354921-12282-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.254.131:0
--local_ip_for_outbound_sockets=127.11.254.131
--webserver_interface=127.11.254.131
--webserver_port=0
--tserver_master_addrs=127.11.254.190:42795
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.131.keytab
--principal=kudu/127.11.254.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.254.148:34211
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568607354921-12282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568607354921-12282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568607354921-12282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:03:34.309010 14881 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:03:34.309481 14881 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:03:34.309567 14881 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:03:34.311803 14881 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:03:34.311911 14881 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:03:34.311980 14881 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 21:03:34.312080 14881 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.254.131
I20260812 21:03:34.315148 14881 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.254.148:34211
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568607354921-12282-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568607354921-12282-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.131.keytab
--principal=kudu/127.11.254.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.254.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568607354921-12282-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.254.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.11.254.190:42795
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14881
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.254.131
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568607354921-12282-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:03:34.315587 14881 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:03:34.316004 14881 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:03:34.325014 14888 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:03:34.325079 14891 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:03:34.325050 14889 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:03:34.325460 14881 server_base.cc:1034] running on GCE node
I20260812 21:03:34.325984 14881 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:03:34.326474 14881 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:03:34.327657 14881 hybrid_clock.cc:648] HybridClock initialized: now 1786568614327621 us; error 45 us; skew 500 ppm
Aug 12 21:03:34 dist-test-slave-vpvm krb5kdc[14170](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568614, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:03:34.337539 14881 init.cc:377] Logged in from keytab as kudu/127.11.254.131@KRBTEST.COM (short username kudu)
I20260812 21:03:34.338508 14881 webserver.cc:460] Webserver started at http://127.11.254.131:44723/ using document root <none> and password file <none>
I20260812 21:03:34.338872 14881 fs_manager.cc:359] Metadata directory not provided
I20260812 21:03:34.338997 14881 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:03:34.339278 14881 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:03:34.341867 14881 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568607354921-12282-0/minicluster-data/ts-2/data/instance:
uuid: "b715cb5f104640da9111c451705b6d78"
format_stamp: "Formatted at 2026-08-12 21:03:34 on dist-test-slave-vpvm"
server_key: "858474a6a34ec21a39a55bef24b28073"
server_key_iv: "5787e046b717a08792bd3b8edc6ffd5a"
server_key_version: "encryptionkey@0"
I20260812 21:03:34.342700 14881 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568607354921-12282-0/minicluster-data/ts-2/wal/instance:
uuid: "b715cb5f104640da9111c451705b6d78"
format_stamp: "Formatted at 2026-08-12 21:03:34 on dist-test-slave-vpvm"
server_key: "858474a6a34ec21a39a55bef24b28073"
server_key_iv: "5787e046b717a08792bd3b8edc6ffd5a"
server_key_version: "encryptionkey@0"
I20260812 21:03:34.347937 14881 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260812 21:03:34.349514 14903 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:03:34.349785 14881 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 21:03:34.350035 14881 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568607354921-12282-0/minicluster-data/ts-2/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568607354921-12282-0/minicluster-data/ts-2/wal
uuid: "b715cb5f104640da9111c451705b6d78"
format_stamp: "Formatted at 2026-08-12 21:03:34 on dist-test-slave-vpvm"
server_key: "858474a6a34ec21a39a55bef24b28073"
server_key_iv: "5787e046b717a08792bd3b8edc6ffd5a"
server_key_version: "encryptionkey@0"
I20260812 21:03:34.350147 14881 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568607354921-12282-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568607354921-12282-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568607354921-12282-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:03:34.367691 14881 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:03:34.373122 14881 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:03:34.373428 14881 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:03:34.374605 14881 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:03:34.374682 14881 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:03:34.374724 14881 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:03:34.374750 14881 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 21:03:34 dist-test-slave-vpvm krb5kdc[14170](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568614, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.131@KRBTEST.COM for kudu/127.11.254.190@KRBTEST.COM
I20260812 21:03:34.387481 14881 rpc_server.cc:307] RPC server started. Bound to: 127.11.254.131:41177
I20260812 21:03:34.387658 15102 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.254.131:41177 every 8 connection(s)
I20260812 21:03:34.388059 14881 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568607354921-12282-0/minicluster-data/ts-2/data/info.pb
I20260812 21:03:34.392139 14332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:34.375186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.131:59207 (local address 127.11.254.190:42795)
0812 21:03:34.375742 (+ 556us) server_negotiation.cc:206] Beginning negotiation
0812 21:03:34.375744 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:03:34.375766 (+ 22us) server_negotiation.cc:407] Connection header received
0812 21:03:34.376550 (+ 784us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:03:34.376552 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:03:34.376578 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:03:34.376705 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:03:34.378563 (+ 1858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:34.379540 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:34.381425 (+ 1885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:34.381816 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:34.381914 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:34.385824 (+ 3910us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:03:34.385825 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:03:34.385826 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:03:34.385844 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:03:34.389518 (+ 3674us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:34.390287 (+ 769us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:34.390289 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:34.390289 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:34.390403 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:34.390914 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:34.390915 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:34.390916 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:34.391207 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:03:34.391330 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0812 21:03:34.391753 (+ 423us) server_negotiation.cc:299] Negotiation successful
0812 21:03:34.391958 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":492}
I20260812 21:03:34.392530 14914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:34.375210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:42795 (local address 127.11.254.131:59207)
0812 21:03:34.375464 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0812 21:03:34.375525 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0812 21:03:34.376405 (+ 880us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:03:34.376941 (+ 536us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:03:34.376954 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:03:34.377788 (+ 834us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:03:34.378382 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:34.378391 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:34.379680 (+ 1289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:34.379681 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:34.381251 (+ 1570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:34.381254 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:34.381930 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:34.381931 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:34.382201 (+ 270us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:34.383010 (+ 809us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:03:34.383036 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:03:34.385645 (+ 2609us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:03:34.389697 (+ 4052us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:03:34.389711 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:03:34.389714 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:03:34.390189 (+ 475us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:03:34.390493 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:03:34.390493 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:03:34.390494 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:03:34.390761 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:03:34.391320 (+ 559us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:03:34.391326 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:03:34.391531 (+ 205us) client_negotiation.cc:769] Sending connection context
0812 21:03:34.391669 (+ 138us) client_negotiation.cc:240] Negotiation successful
0812 21:03:34.392073 (+ 404us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":120,"threads_started":1}
I20260812 21:03:34.393461 15105 heartbeater.cc:347] Connected to a master server at 127.11.254.190:42795
I20260812 21:03:34.393649 15105 heartbeater.cc:464] Registering TS with master...
I20260812 21:03:34.394145 15105 heartbeater.cc:511] Master 127.11.254.190:42795 requested a full tablet report, sending...
I20260812 21:03:34.394874 14247 ts_manager.cc:194] Registered new tserver with Master: b715cb5f104640da9111c451705b6d78 (127.11.254.131:41177)
I20260812 21:03:34.395480 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 14881
I20260812 21:03:34.395635 12282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568607354921-12282-0/minicluster-data/ts-2/wal/instance
I20260812 21:03:34.395728 14247 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.254.131@KRBTEST.COM'} at 127.11.254.131:59207
I20260812 21:03:34.395959 12282 external_mini_cluster.cc:1428] Setting key afae5e8c8964e830138f71c50e98aa59
I20260812 21:03:34.397547 12282 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 21:03:34.406318 15117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:34.395320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:42795 (local address 127.11.254.131:34611)
0812 21:03:34.395692 (+ 372us) negotiation.cc:107] Waiting for socket to connect
0812 21:03:34.395703 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 21:03:34.395825 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:03:34.396126 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:03:34.396127 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:03:34.396861 (+ 734us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:03:34.397028 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:34.397032 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:34.398372 (+ 1340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:34.398379 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:34.399345 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:34.399348 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:34.400001 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:34.400002 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:34.400144 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:34.401043 (+ 899us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:03:34.401049 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:03:34.401625 (+ 576us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:03:34.404275 (+ 2650us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:03:34.404276 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:03:34.404277 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:03:34.404679 (+ 402us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:03:34.405057 (+ 378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:03:34.405061 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:03:34.405062 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:03:34.405167 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:03:34.405685 (+ 518us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:03:34.405686 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:03:34.405771 (+ 85us) client_negotiation.cc:769] Sending connection context
0812 21:03:34.405869 (+ 98us) client_negotiation.cc:240] Negotiation successful
0812 21:03:34.406123 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":115,"threads_started":1}
I20260812 21:03:34.406997 14332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:34.395517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.131:34611 (local address 127.11.254.190:42795)
0812 21:03:34.395607 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0812 21:03:34.395610 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 21:03:34.395772 (+ 162us) server_negotiation.cc:407] Connection header received
0812 21:03:34.395926 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:03:34.395927 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:03:34.395972 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:03:34.396138 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:03:34.397250 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:34.398244 (+ 994us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:34.399473 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:34.399871 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:34.400019 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:34.401737 (+ 1718us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:03:34.401738 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:03:34.401738 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:03:34.401754 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:03:34.404106 (+ 2352us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:34.404816 (+ 710us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:34.404818 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:34.404818 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:34.404941 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:34.405293 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:34.405294 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:34.405294 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:34.405531 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:03:34.405646 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0812 21:03:34.406793 (+ 1147us) server_negotiation.cc:299] Negotiation successful
0812 21:03:34.406898 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
Aug 12 21:03:34 dist-test-slave-vpvm krb5kdc[14170](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568614, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 21:03:34 dist-test-slave-vpvm krb5kdc[14170](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568614, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.11.254.190@KRBTEST.COM
I20260812 21:03:34.436851 14332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:34.426910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40146 (local address 127.11.254.190:42795)
0812 21:03:34.426973 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0812 21:03:34.426975 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:03:34.426995 (+ 20us) server_negotiation.cc:407] Connection header received
0812 21:03:34.427163 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:03:34.427164 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:03:34.427181 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:03:34.427255 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:03:34.428805 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:34.429604 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:34.430661 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:34.430914 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:34.430995 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:34.433853 (+ 2858us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:03:34.433855 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:03:34.433855 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:03:34.433864 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:03:34.435145 (+ 1281us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:34.435648 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:34.435649 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:34.435650 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:34.435721 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:34.435971 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:34.435971 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:34.435972 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:34.436143 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:03:34.436287 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0812 21:03:34.436496 (+ 209us) server_negotiation.cc:299] Negotiation successful
0812 21:03:34.436689 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 21:03:34.439154 14247 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40146:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 21:03:34.442239 14247 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 21:03:34.452955 15146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:34.446647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43420 (local address 127.11.254.130:45471)
0812 21:03:34.446987 (+ 340us) server_negotiation.cc:206] Beginning negotiation
0812 21:03:34.446992 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 21:03:34.447002 (+ 10us) server_negotiation.cc:407] Connection header received
0812 21:03:34.447080 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:03:34.447082 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:03:34.447114 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:03:34.447215 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:03:34.447563 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:34.449299 (+ 1736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:34.451131 (+ 1832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:34.452165 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:34.452316 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:34.452421 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0812 21:03:34.452709 (+ 288us) server_negotiation.cc:299] Negotiation successful
0812 21:03:34.452780 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":276,"thread_start_us":149,"threads_started":1}
I20260812 21:03:34.452946 15143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:34.446052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.130:45471 (local address 127.0.0.1:43420)
0812 21:03:34.446847 (+ 795us) negotiation.cc:107] Waiting for socket to connect
0812 21:03:34.446867 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 21:03:34.446969 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:03:34.447278 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:03:34.447292 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:03:34.447295 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:03:34.447411 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:34.447423 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:34.449527 (+ 2104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:34.449528 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:34.450974 (+ 1446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:34.450976 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:34.452315 (+ 1339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:34.452316 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:34.452490 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:34.452525 (+ 35us) client_negotiation.cc:769] Sending connection context
0812 21:03:34.452682 (+ 157us) client_negotiation.cc:240] Negotiation successful
0812 21:03:34.452793 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":745,"mutex_wait_us":57,"thread_start_us":192,"threads_started":1}
I20260812 21:03:34.453394 15140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:34.445934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.129:42771 (local address 127.0.0.1:37766)
0812 21:03:34.446203 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0812 21:03:34.446285 (+ 82us) client_negotiation.cc:174] Beginning negotiation
0812 21:03:34.446579 (+ 294us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:03:34.447959 (+ 1380us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:03:34.447960 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:03:34.447963 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:03:34.448030 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:34.448033 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:34.449860 (+ 1827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:34.449862 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:34.451537 (+ 1675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:34.451548 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:34.453078 (+ 1530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:34.453079 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:34.453170 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:34.453180 (+ 10us) client_negotiation.cc:769] Sending connection context
0812 21:03:34.453246 (+ 66us) client_negotiation.cc:240] Negotiation successful
0812 21:03:34.453304 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":126,"threads_started":1}
I20260812 21:03:34.453773 15142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:34.446333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37766 (local address 127.11.254.129:42771)
0812 21:03:34.447673 (+ 1340us) server_negotiation.cc:206] Beginning negotiation
0812 21:03:34.447677 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 21:03:34.447685 (+ 8us) server_negotiation.cc:407] Connection header received
0812 21:03:34.447752 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:03:34.447754 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:03:34.447784 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:03:34.447961 (+ 177us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:03:34.448203 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:34.449726 (+ 1523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:34.451708 (+ 1982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:34.452819 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:34.452939 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:34.453027 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0812 21:03:34.453573 (+ 546us) server_negotiation.cc:299] Negotiation successful
0812 21:03:34.453670 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1292,"thread_start_us":133,"threads_started":1}
I20260812 21:03:34.454203 15141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:34.446227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.131:41177 (local address 127.0.0.1:46312)
0812 21:03:34.446932 (+ 705us) negotiation.cc:107] Waiting for socket to connect
0812 21:03:34.446951 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 21:03:34.447007 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:03:34.447477 (+ 470us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:03:34.447479 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:03:34.447489 (+ 10us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:03:34.447594 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:34.447604 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:34.449442 (+ 1838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:34.449444 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:34.452067 (+ 2623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:34.452069 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:34.453735 (+ 1666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:34.453736 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:34.453837 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:34.453854 (+ 17us) client_negotiation.cc:769] Sending connection context
0812 21:03:34.453987 (+ 133us) client_negotiation.cc:240] Negotiation successful
0812 21:03:34.454053 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":677,"mutex_wait_us":113,"thread_start_us":253,"threads_started":1}
I20260812 21:03:34.454532 14746 tablet_service.cc:1467] Processing CreateTablet for tablet b92996e25ee74e18a6e2cbb46cb00399 (DEFAULT_TABLE table=test-table [id=0b07dad266f34e878498f74c30938e76]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:03:34.454857 15144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:34.446577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46312 (local address 127.11.254.131:41177)
0812 21:03:34.447237 (+ 660us) server_negotiation.cc:206] Beginning negotiation
0812 21:03:34.447242 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 21:03:34.447264 (+ 22us) server_negotiation.cc:407] Connection header received
0812 21:03:34.447328 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:03:34.447331 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:03:34.447348 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:03:34.447490 (+ 142us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:03:34.447777 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:34.449286 (+ 1509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:34.452221 (+ 2935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:34.453423 (+ 1202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:34.453567 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:34.453675 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0812 21:03:34.454599 (+ 924us) server_negotiation.cc:299] Negotiation successful
0812 21:03:34.454682 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":602,"thread_start_us":184,"threads_started":1}
I20260812 21:03:34.455207 14746 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b92996e25ee74e18a6e2cbb46cb00399. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:03:34.455953 14992 tablet_service.cc:1467] Processing CreateTablet for tablet b92996e25ee74e18a6e2cbb46cb00399 (DEFAULT_TABLE table=test-table [id=0b07dad266f34e878498f74c30938e76]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:03:34.456642 14992 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b92996e25ee74e18a6e2cbb46cb00399. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:03:34.459118 14501 tablet_service.cc:1467] Processing CreateTablet for tablet b92996e25ee74e18a6e2cbb46cb00399 (DEFAULT_TABLE table=test-table [id=0b07dad266f34e878498f74c30938e76]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:03:34.459892 14501 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b92996e25ee74e18a6e2cbb46cb00399. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:03:34.462356 15147 tablet_bootstrap.cc:492] T b92996e25ee74e18a6e2cbb46cb00399 P ef41ad9d59be4951885e7143db439d06: Bootstrap starting.
I20260812 21:03:34.464236 15147 tablet_bootstrap.cc:654] T b92996e25ee74e18a6e2cbb46cb00399 P ef41ad9d59be4951885e7143db439d06: Neither blocks nor log segments found. Creating new log.
I20260812 21:03:34.465058 15147 log.cc:824] T b92996e25ee74e18a6e2cbb46cb00399 P ef41ad9d59be4951885e7143db439d06: Log is configured to *not* fsync() on all Append() calls
I20260812 21:03:34.466128 15148 tablet_bootstrap.cc:492] T b92996e25ee74e18a6e2cbb46cb00399 P e619668bf91d4868bc386cbf6f07c3a2: Bootstrap starting.
I20260812 21:03:34.466627 15147 tablet_bootstrap.cc:492] T b92996e25ee74e18a6e2cbb46cb00399 P ef41ad9d59be4951885e7143db439d06: No bootstrap required, opened a new log
I20260812 21:03:34.466733 15147 ts_tablet_manager.cc:1397] T b92996e25ee74e18a6e2cbb46cb00399 P ef41ad9d59be4951885e7143db439d06: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260812 21:03:34.467154 15154 tablet_bootstrap.cc:492] T b92996e25ee74e18a6e2cbb46cb00399 P b715cb5f104640da9111c451705b6d78: Bootstrap starting.
I20260812 21:03:34.467455 15148 tablet_bootstrap.cc:654] T b92996e25ee74e18a6e2cbb46cb00399 P e619668bf91d4868bc386cbf6f07c3a2: Neither blocks nor log segments found. Creating new log.
I20260812 21:03:34.467944 15148 log.cc:824] T b92996e25ee74e18a6e2cbb46cb00399 P e619668bf91d4868bc386cbf6f07c3a2: Log is configured to *not* fsync() on all Append() calls
I20260812 21:03:34.468968 15154 tablet_bootstrap.cc:654] T b92996e25ee74e18a6e2cbb46cb00399 P b715cb5f104640da9111c451705b6d78: Neither blocks nor log segments found. Creating new log.
I20260812 21:03:34.469165 15148 tablet_bootstrap.cc:492] T b92996e25ee74e18a6e2cbb46cb00399 P e619668bf91d4868bc386cbf6f07c3a2: No bootstrap required, opened a new log
I20260812 21:03:34.469285 15148 ts_tablet_manager.cc:1397] T b92996e25ee74e18a6e2cbb46cb00399 P e619668bf91d4868bc386cbf6f07c3a2: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260812 21:03:34.469565 15154 log.cc:824] T b92996e25ee74e18a6e2cbb46cb00399 P b715cb5f104640da9111c451705b6d78: Log is configured to *not* fsync() on all Append() calls
I20260812 21:03:34.470322 15147 raft_consensus.cc:348] T b92996e25ee74e18a6e2cbb46cb00399 P ef41ad9d59be4951885e7143db439d06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e619668bf91d4868bc386cbf6f07c3a2" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 42771 } } peers { permanent_uuid: "ef41ad9d59be4951885e7143db439d06" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 45471 } } peers { permanent_uuid: "b715cb5f104640da9111c451705b6d78" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 41177 } }
I20260812 21:03:34.470568 15147 raft_consensus.cc:374] T b92996e25ee74e18a6e2cbb46cb00399 P ef41ad9d59be4951885e7143db439d06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:03:34.470659 15147 raft_consensus.cc:729] T b92996e25ee74e18a6e2cbb46cb00399 P ef41ad9d59be4951885e7143db439d06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef41ad9d59be4951885e7143db439d06, State: Initialized, Role: FOLLOWER
I20260812 21:03:34.470873 15147 consensus_queue.cc:260] T b92996e25ee74e18a6e2cbb46cb00399 P ef41ad9d59be4951885e7143db439d06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e619668bf91d4868bc386cbf6f07c3a2" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 42771 } } peers { permanent_uuid: "ef41ad9d59be4951885e7143db439d06" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 45471 } } peers { permanent_uuid: "b715cb5f104640da9111c451705b6d78" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 41177 } }
I20260812 21:03:34.471052 15154 tablet_bootstrap.cc:492] T b92996e25ee74e18a6e2cbb46cb00399 P b715cb5f104640da9111c451705b6d78: No bootstrap required, opened a new log
I20260812 21:03:34.471177 15154 ts_tablet_manager.cc:1397] T b92996e25ee74e18a6e2cbb46cb00399 P b715cb5f104640da9111c451705b6d78: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.002s
I20260812 21:03:34.471290 15147 ts_tablet_manager.cc:1428] T b92996e25ee74e18a6e2cbb46cb00399 P ef41ad9d59be4951885e7143db439d06: Time spent starting tablet: real 0.004s user 0.005s sys 0.000s
I20260812 21:03:34.471470 14864 heartbeater.cc:503] Master 127.11.254.190:42795 was elected leader, sending a full tablet report...
I20260812 21:03:34.473521 15148 raft_consensus.cc:348] T b92996e25ee74e18a6e2cbb46cb00399 P e619668bf91d4868bc386cbf6f07c3a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e619668bf91d4868bc386cbf6f07c3a2" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 42771 } } peers { permanent_uuid: "ef41ad9d59be4951885e7143db439d06" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 45471 } } peers { permanent_uuid: "b715cb5f104640da9111c451705b6d78" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 41177 } }
I20260812 21:03:34.473747 15148 raft_consensus.cc:374] T b92996e25ee74e18a6e2cbb46cb00399 P e619668bf91d4868bc386cbf6f07c3a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:03:34.473829 15148 raft_consensus.cc:729] T b92996e25ee74e18a6e2cbb46cb00399 P e619668bf91d4868bc386cbf6f07c3a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e619668bf91d4868bc386cbf6f07c3a2, State: Initialized, Role: FOLLOWER
I20260812 21:03:34.474112 15148 consensus_queue.cc:260] T b92996e25ee74e18a6e2cbb46cb00399 P e619668bf91d4868bc386cbf6f07c3a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e619668bf91d4868bc386cbf6f07c3a2" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 42771 } } peers { permanent_uuid: "ef41ad9d59be4951885e7143db439d06" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 45471 } } peers { permanent_uuid: "b715cb5f104640da9111c451705b6d78" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 41177 } }
I20260812 21:03:34.474507 15148 ts_tablet_manager.cc:1428] T b92996e25ee74e18a6e2cbb46cb00399 P e619668bf91d4868bc386cbf6f07c3a2: Time spent starting tablet: real 0.005s user 0.005s sys 0.001s
I20260812 21:03:34.474613 14622 heartbeater.cc:503] Master 127.11.254.190:42795 was elected leader, sending a full tablet report...
I20260812 21:03:34.475744 15154 raft_consensus.cc:348] T b92996e25ee74e18a6e2cbb46cb00399 P b715cb5f104640da9111c451705b6d78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e619668bf91d4868bc386cbf6f07c3a2" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 42771 } } peers { permanent_uuid: "ef41ad9d59be4951885e7143db439d06" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 45471 } } peers { permanent_uuid: "b715cb5f104640da9111c451705b6d78" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 41177 } }
I20260812 21:03:34.476109 15154 raft_consensus.cc:374] T b92996e25ee74e18a6e2cbb46cb00399 P b715cb5f104640da9111c451705b6d78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:03:34.476238 15154 raft_consensus.cc:729] T b92996e25ee74e18a6e2cbb46cb00399 P b715cb5f104640da9111c451705b6d78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b715cb5f104640da9111c451705b6d78, State: Initialized, Role: FOLLOWER
I20260812 21:03:34.476541 15154 consensus_queue.cc:260] T b92996e25ee74e18a6e2cbb46cb00399 P b715cb5f104640da9111c451705b6d78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e619668bf91d4868bc386cbf6f07c3a2" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 42771 } } peers { permanent_uuid: "ef41ad9d59be4951885e7143db439d06" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 45471 } } peers { permanent_uuid: "b715cb5f104640da9111c451705b6d78" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 41177 } }
I20260812 21:03:34.477142 15154 ts_tablet_manager.cc:1428] T b92996e25ee74e18a6e2cbb46cb00399 P b715cb5f104640da9111c451705b6d78: Time spent starting tablet: real 0.006s user 0.003s sys 0.002s
I20260812 21:03:34.477294 15105 heartbeater.cc:503] Master 127.11.254.190:42795 was elected leader, sending a full tablet report...
I20260812 21:03:34.493522 15160 raft_consensus.cc:482] T b92996e25ee74e18a6e2cbb46cb00399 P e619668bf91d4868bc386cbf6f07c3a2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 21:03:34.493758 15160 raft_consensus.cc:504] T b92996e25ee74e18a6e2cbb46cb00399 P e619668bf91d4868bc386cbf6f07c3a2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e619668bf91d4868bc386cbf6f07c3a2" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 42771 } } peers { permanent_uuid: "ef41ad9d59be4951885e7143db439d06" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 45471 } } peers { permanent_uuid: "b715cb5f104640da9111c451705b6d78" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 41177 } }
I20260812 21:03:34.494372 15160 leader_election.cc:288] T b92996e25ee74e18a6e2cbb46cb00399 P e619668bf91d4868bc386cbf6f07c3a2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ef41ad9d59be4951885e7143db439d06 (127.11.254.130:45471), b715cb5f104640da9111c451705b6d78 (127.11.254.131:41177)
I20260812 21:03:34.498248 15146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:34.494807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.129:42719 (local address 127.11.254.130:45471)
0812 21:03:34.494878 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0812 21:03:34.494879 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 21:03:34.495300 (+ 421us) server_negotiation.cc:407] Connection header received
0812 21:03:34.495414 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:03:34.495419 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:03:34.495435 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:03:34.495499 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:03:34.495672 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:34.496162 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:34.497226 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:34.497725 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:34.497813 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:34.497831 (+ 18us) server_negotiation.cc:1035] Waiting for connection context
0812 21:03:34.498074 (+ 243us) server_negotiation.cc:299] Negotiation successful
0812 21:03:34.498139 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 21:03:34.498314 15144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:34.494674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.129:37121 (local address 127.11.254.131:41177)
0812 21:03:34.494745 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0812 21:03:34.494748 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 21:03:34.495162 (+ 414us) server_negotiation.cc:407] Connection header received
0812 21:03:34.495282 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:03:34.495283 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:03:34.495298 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:03:34.495367 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:03:34.495589 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:34.496177 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:34.497231 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:34.497798 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:34.497899 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:34.497909 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0812 21:03:34.498125 (+ 216us) server_negotiation.cc:299] Negotiation successful
0812 21:03:34.498196 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260812 21:03:34.498332 15163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:34.494581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.131:41177 (local address 127.11.254.129:37121)
0812 21:03:34.495054 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0812 21:03:34.495069 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 21:03:34.495174 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:03:34.495404 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:03:34.495405 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:03:34.495408 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:03:34.495517 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:34.495519 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:34.496259 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:34.496260 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:34.497135 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:34.497138 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:34.497900 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:34.497901 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:34.498034 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:34.498039 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 21:03:34.498126 (+ 87us) client_negotiation.cc:240] Negotiation successful
0812 21:03:34.498196 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":448,"mutex_wait_us":24,"thread_start_us":528,"threads_started":1}
I20260812 21:03:34.498616 15164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:34.494789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.130:45471 (local address 127.11.254.129:42719)
0812 21:03:34.495201 (+ 412us) negotiation.cc:107] Waiting for socket to connect
0812 21:03:34.495219 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 21:03:34.495314 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:03:34.495522 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:03:34.495523 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:03:34.495525 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:03:34.495615 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:34.495616 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:34.496243 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:34.496244 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:34.497137 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:34.497140 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:34.497844 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:34.497846 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:34.497936 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:34.497940 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 21:03:34.498058 (+ 118us) client_negotiation.cc:240] Negotiation successful
0812 21:03:34.498125 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":384,"mutex_wait_us":59,"thread_start_us":166,"threads_started":1}
I20260812 21:03:34.498842 15032 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b92996e25ee74e18a6e2cbb46cb00399" candidate_uuid: "e619668bf91d4868bc386cbf6f07c3a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b715cb5f104640da9111c451705b6d78" is_pre_election: true
I20260812 21:03:34.499022 14781 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b92996e25ee74e18a6e2cbb46cb00399" candidate_uuid: "e619668bf91d4868bc386cbf6f07c3a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef41ad9d59be4951885e7143db439d06" is_pre_election: true
I20260812 21:03:34.499068 15032 raft_consensus.cc:2454] T b92996e25ee74e18a6e2cbb46cb00399 P b715cb5f104640da9111c451705b6d78 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e619668bf91d4868bc386cbf6f07c3a2 in term 0.
I20260812 21:03:34.499220 14781 raft_consensus.cc:2454] T b92996e25ee74e18a6e2cbb46cb00399 P ef41ad9d59be4951885e7143db439d06 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e619668bf91d4868bc386cbf6f07c3a2 in term 0.
I20260812 21:03:34.499437 14414 leader_election.cc:302] T b92996e25ee74e18a6e2cbb46cb00399 P e619668bf91d4868bc386cbf6f07c3a2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b715cb5f104640da9111c451705b6d78, e619668bf91d4868bc386cbf6f07c3a2; no voters:
I20260812 21:03:34.499672 15160 raft_consensus.cc:2781] T b92996e25ee74e18a6e2cbb46cb00399 P e619668bf91d4868bc386cbf6f07c3a2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 21:03:34.499774 15160 raft_consensus.cc:482] T b92996e25ee74e18a6e2cbb46cb00399 P e619668bf91d4868bc386cbf6f07c3a2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 21:03:34.499812 15160 raft_consensus.cc:3037] T b92996e25ee74e18a6e2cbb46cb00399 P e619668bf91d4868bc386cbf6f07c3a2 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:03:34.501269 15160 raft_consensus.cc:504] T b92996e25ee74e18a6e2cbb46cb00399 P e619668bf91d4868bc386cbf6f07c3a2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e619668bf91d4868bc386cbf6f07c3a2" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 42771 } } peers { permanent_uuid: "ef41ad9d59be4951885e7143db439d06" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 45471 } } peers { permanent_uuid: "b715cb5f104640da9111c451705b6d78" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 41177 } }
I20260812 21:03:34.501412 15160 leader_election.cc:288] T b92996e25ee74e18a6e2cbb46cb00399 P e619668bf91d4868bc386cbf6f07c3a2 [CANDIDATE]: Term 1 election: Requested vote from peers ef41ad9d59be4951885e7143db439d06 (127.11.254.130:45471), b715cb5f104640da9111c451705b6d78 (127.11.254.131:41177)
I20260812 21:03:34.501621 14781 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b92996e25ee74e18a6e2cbb46cb00399" candidate_uuid: "e619668bf91d4868bc386cbf6f07c3a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef41ad9d59be4951885e7143db439d06"
I20260812 21:03:34.501729 14781 raft_consensus.cc:3037] T b92996e25ee74e18a6e2cbb46cb00399 P ef41ad9d59be4951885e7143db439d06 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:03:34.502051 15032 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b92996e25ee74e18a6e2cbb46cb00399" candidate_uuid: "e619668bf91d4868bc386cbf6f07c3a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b715cb5f104640da9111c451705b6d78"
I20260812 21:03:34.502168 15032 raft_consensus.cc:3037] T b92996e25ee74e18a6e2cbb46cb00399 P b715cb5f104640da9111c451705b6d78 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:03:34.503603 14781 raft_consensus.cc:2454] T b92996e25ee74e18a6e2cbb46cb00399 P ef41ad9d59be4951885e7143db439d06 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e619668bf91d4868bc386cbf6f07c3a2 in term 1.
I20260812 21:03:34.503602 15032 raft_consensus.cc:2454] T b92996e25ee74e18a6e2cbb46cb00399 P b715cb5f104640da9111c451705b6d78 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e619668bf91d4868bc386cbf6f07c3a2 in term 1.
I20260812 21:03:34.503886 14414 leader_election.cc:302] T b92996e25ee74e18a6e2cbb46cb00399 P e619668bf91d4868bc386cbf6f07c3a2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b715cb5f104640da9111c451705b6d78, e619668bf91d4868bc386cbf6f07c3a2; no voters:
I20260812 21:03:34.504026 15160 raft_consensus.cc:2781] T b92996e25ee74e18a6e2cbb46cb00399 P e619668bf91d4868bc386cbf6f07c3a2 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:03:34.504293 15160 raft_consensus.cc:686] T b92996e25ee74e18a6e2cbb46cb00399 P e619668bf91d4868bc386cbf6f07c3a2 [term 1 LEADER]: Becoming Leader. State: Replica: e619668bf91d4868bc386cbf6f07c3a2, State: Running, Role: LEADER
I20260812 21:03:34.504469 15160 consensus_queue.cc:237] T b92996e25ee74e18a6e2cbb46cb00399 P e619668bf91d4868bc386cbf6f07c3a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e619668bf91d4868bc386cbf6f07c3a2" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 42771 } } peers { permanent_uuid: "ef41ad9d59be4951885e7143db439d06" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 45471 } } peers { permanent_uuid: "b715cb5f104640da9111c451705b6d78" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 41177 } }
I20260812 21:03:34.505861 14246 catalog_manager.cc:5495] T b92996e25ee74e18a6e2cbb46cb00399 P e619668bf91d4868bc386cbf6f07c3a2 reported cstate change: term changed from 0 to 1, leader changed from <none> to e619668bf91d4868bc386cbf6f07c3a2 (127.11.254.129). New cstate: current_term: 1 leader_uuid: "e619668bf91d4868bc386cbf6f07c3a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e619668bf91d4868bc386cbf6f07c3a2" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 42771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef41ad9d59be4951885e7143db439d06" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 45471 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b715cb5f104640da9111c451705b6d78" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 41177 } health_report { overall_health: UNKNOWN } } }
I20260812 21:03:34.531019 15142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:34.527359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37782 (local address 127.11.254.129:42771)
0812 21:03:34.527476 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0812 21:03:34.527478 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:03:34.527503 (+ 25us) server_negotiation.cc:407] Connection header received
0812 21:03:34.527634 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:03:34.527636 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:03:34.527679 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:03:34.527761 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 21:03:34.527988 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:34.528674 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:34.529730 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:34.530062 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:34.530165 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:34.530488 (+ 323us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 21:03:34.530663 (+ 175us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 21:03:34.530777 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0812 21:03:34.530858 (+ 81us) server_negotiation.cc:299] Negotiation successful
0812 21:03:34.530894 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 21:03:34.533828 14781 raft_consensus.cc:1261] T b92996e25ee74e18a6e2cbb46cb00399 P ef41ad9d59be4951885e7143db439d06 [term 1 FOLLOWER]: Refusing update from remote peer e619668bf91d4868bc386cbf6f07c3a2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 21:03:34.533828 15032 raft_consensus.cc:1261] T b92996e25ee74e18a6e2cbb46cb00399 P b715cb5f104640da9111c451705b6d78 [term 1 FOLLOWER]: Refusing update from remote peer e619668bf91d4868bc386cbf6f07c3a2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 21:03:34.534467 15165 consensus_queue.cc:1035] T b92996e25ee74e18a6e2cbb46cb00399 P e619668bf91d4868bc386cbf6f07c3a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b715cb5f104640da9111c451705b6d78" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 41177 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 21:03:34.534711 15160 consensus_queue.cc:1035] T b92996e25ee74e18a6e2cbb46cb00399 P e619668bf91d4868bc386cbf6f07c3a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef41ad9d59be4951885e7143db439d06" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 45471 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 21:03:34.540024 15174 mvcc.cc:205] Tried to move back new op lower bound from 7317785045126254592 to 7317785045011214336. Current Snapshot: MvccSnapshot[applied={T|T < 7317785045126254592}]
I20260812 21:03:34.541425 15176 mvcc.cc:205] Tried to move back new op lower bound from 7317785045126254592 to 7317785045011214336. Current Snapshot: MvccSnapshot[applied={T|T < 7317785045126254592}]
I20260812 21:03:34.547909 14245 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40146:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 21:03:34.548111 14245 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40146:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 21:03:34.549474 14245 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f45b7349c97e46c59225cf0ba1c645e0: Sending DeleteTablet for 3 replicas of tablet b92996e25ee74e18a6e2cbb46cb00399
I20260812 21:03:34.550081 14992 tablet_service.cc:1514] Processing DeleteTablet for tablet b92996e25ee74e18a6e2cbb46cb00399 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:03:34 UTC) from {username='kudu', principal='kudu/127.11.254.190@KRBTEST.COM'} at 127.0.0.1:46312
I20260812 21:03:34.550134 14746 tablet_service.cc:1514] Processing DeleteTablet for tablet b92996e25ee74e18a6e2cbb46cb00399 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:03:34 UTC) from {username='kudu', principal='kudu/127.11.254.190@KRBTEST.COM'} at 127.0.0.1:43420
I20260812 21:03:34.550549 14501 tablet_service.cc:1514] Processing DeleteTablet for tablet b92996e25ee74e18a6e2cbb46cb00399 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:03:34 UTC) from {username='kudu', principal='kudu/127.11.254.190@KRBTEST.COM'} at 127.0.0.1:37766
I20260812 21:03:34.550635 15189 tablet_replica.cc:330] T b92996e25ee74e18a6e2cbb46cb00399 P b715cb5f104640da9111c451705b6d78: stopping tablet replica
I20260812 21:03:34.550906 15189 raft_consensus.cc:2229] T b92996e25ee74e18a6e2cbb46cb00399 P b715cb5f104640da9111c451705b6d78 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 21:03:34.550940 14245 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40146:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260812 21:03:34.551129 15191 tablet_replica.cc:330] T b92996e25ee74e18a6e2cbb46cb00399 P ef41ad9d59be4951885e7143db439d06: stopping tablet replica
W20260812 21:03:34.551205 14245 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 21:03:34.551301 15191 raft_consensus.cc:2229] T b92996e25ee74e18a6e2cbb46cb00399 P ef41ad9d59be4951885e7143db439d06 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 21:03:34.551546 15191 raft_consensus.cc:2258] T b92996e25ee74e18a6e2cbb46cb00399 P ef41ad9d59be4951885e7143db439d06 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 21:03:34.551565 15189 raft_consensus.cc:2258] T b92996e25ee74e18a6e2cbb46cb00399 P b715cb5f104640da9111c451705b6d78 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 21:03:34.552448 15192 tablet_replica.cc:330] T b92996e25ee74e18a6e2cbb46cb00399 P e619668bf91d4868bc386cbf6f07c3a2: stopping tablet replica
I20260812 21:03:34.552740 15192 raft_consensus.cc:2229] T b92996e25ee74e18a6e2cbb46cb00399 P e619668bf91d4868bc386cbf6f07c3a2 [term 1 LEADER]: Raft consensus shutting down.
I20260812 21:03:34.553051 15192 raft_consensus.cc:2258] T b92996e25ee74e18a6e2cbb46cb00399 P e619668bf91d4868bc386cbf6f07c3a2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 21:03:34.553710 14501 tablet_service.cc:1467] Processing CreateTablet for tablet 4dc654afcafe4a32bba79b48a707d3c5 (DEFAULT_TABLE table=test-table [id=bd60207299a94d0cb877232316eab99b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:03:34.553911 14501 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4dc654afcafe4a32bba79b48a707d3c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:03:34.554093 15192 ts_tablet_manager.cc:1905] T b92996e25ee74e18a6e2cbb46cb00399 P e619668bf91d4868bc386cbf6f07c3a2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 21:03:34.554423 14992 tablet_service.cc:1467] Processing CreateTablet for tablet 4dc654afcafe4a32bba79b48a707d3c5 (DEFAULT_TABLE table=test-table [id=bd60207299a94d0cb877232316eab99b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:03:34.554622 14992 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4dc654afcafe4a32bba79b48a707d3c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:03:34.556694 15148 tablet_bootstrap.cc:492] T 4dc654afcafe4a32bba79b48a707d3c5 P e619668bf91d4868bc386cbf6f07c3a2: Bootstrap starting.
I20260812 21:03:34.557416 14746 tablet_service.cc:1467] Processing CreateTablet for tablet 4dc654afcafe4a32bba79b48a707d3c5 (DEFAULT_TABLE table=test-table [id=bd60207299a94d0cb877232316eab99b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:03:34.557463 15154 tablet_bootstrap.cc:492] T 4dc654afcafe4a32bba79b48a707d3c5 P b715cb5f104640da9111c451705b6d78: Bootstrap starting.
I20260812 21:03:34.557639 14746 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4dc654afcafe4a32bba79b48a707d3c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:03:34.557853 15189 ts_tablet_manager.cc:1905] T b92996e25ee74e18a6e2cbb46cb00399 P b715cb5f104640da9111c451705b6d78: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 21:03:34.558296 15148 tablet_bootstrap.cc:654] T 4dc654afcafe4a32bba79b48a707d3c5 P e619668bf91d4868bc386cbf6f07c3a2: Neither blocks nor log segments found. Creating new log.
I20260812 21:03:34.558907 15154 tablet_bootstrap.cc:654] T 4dc654afcafe4a32bba79b48a707d3c5 P b715cb5f104640da9111c451705b6d78: Neither blocks nor log segments found. Creating new log.
I20260812 21:03:34.560631 15147 tablet_bootstrap.cc:492] T 4dc654afcafe4a32bba79b48a707d3c5 P ef41ad9d59be4951885e7143db439d06: Bootstrap starting.
I20260812 21:03:34.560724 15192 ts_tablet_manager.cc:1918] T b92996e25ee74e18a6e2cbb46cb00399 P e619668bf91d4868bc386cbf6f07c3a2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 21:03:34.560580 15189 ts_tablet_manager.cc:1918] T b92996e25ee74e18a6e2cbb46cb00399 P b715cb5f104640da9111c451705b6d78: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 21:03:34.560799 15192 log.cc:1196] T b92996e25ee74e18a6e2cbb46cb00399 P e619668bf91d4868bc386cbf6f07c3a2: Deleting WAL directory at /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568607354921-12282-0/minicluster-data/ts-0/wal/wals/b92996e25ee74e18a6e2cbb46cb00399
I20260812 21:03:34.560820 15189 log.cc:1196] T b92996e25ee74e18a6e2cbb46cb00399 P b715cb5f104640da9111c451705b6d78: Deleting WAL directory at /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568607354921-12282-0/minicluster-data/ts-2/wal/wals/b92996e25ee74e18a6e2cbb46cb00399
I20260812 21:03:34.561120 15189 ts_tablet_manager.cc:1939] T b92996e25ee74e18a6e2cbb46cb00399 P b715cb5f104640da9111c451705b6d78: Deleting consensus metadata
I20260812 21:03:34.561125 15192 ts_tablet_manager.cc:1939] T b92996e25ee74e18a6e2cbb46cb00399 P e619668bf91d4868bc386cbf6f07c3a2: Deleting consensus metadata
I20260812 21:03:34.561586 14219 catalog_manager.cc:4841] TS b715cb5f104640da9111c451705b6d78 (127.11.254.131:41177): tablet b92996e25ee74e18a6e2cbb46cb00399 (table test-table [id=0b07dad266f34e878498f74c30938e76]) successfully deleted
I20260812 21:03:34.561676 15147 tablet_bootstrap.cc:654] T 4dc654afcafe4a32bba79b48a707d3c5 P ef41ad9d59be4951885e7143db439d06: Neither blocks nor log segments found. Creating new log.
I20260812 21:03:34.561821 15148 tablet_bootstrap.cc:492] T 4dc654afcafe4a32bba79b48a707d3c5 P e619668bf91d4868bc386cbf6f07c3a2: No bootstrap required, opened a new log
I20260812 21:03:34.561863 15148 ts_tablet_manager.cc:1397] T 4dc654afcafe4a32bba79b48a707d3c5 P e619668bf91d4868bc386cbf6f07c3a2: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20260812 21:03:34.562036 15191 ts_tablet_manager.cc:1905] T b92996e25ee74e18a6e2cbb46cb00399 P ef41ad9d59be4951885e7143db439d06: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 21:03:34.562194 15148 raft_consensus.cc:348] T 4dc654afcafe4a32bba79b48a707d3c5 P e619668bf91d4868bc386cbf6f07c3a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e619668bf91d4868bc386cbf6f07c3a2" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 42771 } } peers { permanent_uuid: "ef41ad9d59be4951885e7143db439d06" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 45471 } } peers { permanent_uuid: "b715cb5f104640da9111c451705b6d78" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 41177 } }
I20260812 21:03:34.562287 15148 raft_consensus.cc:374] T 4dc654afcafe4a32bba79b48a707d3c5 P e619668bf91d4868bc386cbf6f07c3a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:03:34.562309 15148 raft_consensus.cc:729] T 4dc654afcafe4a32bba79b48a707d3c5 P e619668bf91d4868bc386cbf6f07c3a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e619668bf91d4868bc386cbf6f07c3a2, State: Initialized, Role: FOLLOWER
I20260812 21:03:34.562377 15148 consensus_queue.cc:260] T 4dc654afcafe4a32bba79b48a707d3c5 P e619668bf91d4868bc386cbf6f07c3a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e619668bf91d4868bc386cbf6f07c3a2" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 42771 } } peers { permanent_uuid: "ef41ad9d59be4951885e7143db439d06" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 45471 } } peers { permanent_uuid: "b715cb5f104640da9111c451705b6d78" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 41177 } }
I20260812 21:03:34.562383 14221 catalog_manager.cc:4841] TS e619668bf91d4868bc386cbf6f07c3a2 (127.11.254.129:42771): tablet b92996e25ee74e18a6e2cbb46cb00399 (table test-table [id=0b07dad266f34e878498f74c30938e76]) successfully deleted
I20260812 21:03:34.562541 15148 ts_tablet_manager.cc:1428] T 4dc654afcafe4a32bba79b48a707d3c5 P e619668bf91d4868bc386cbf6f07c3a2: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260812 21:03:34.563920 15191 ts_tablet_manager.cc:1918] T b92996e25ee74e18a6e2cbb46cb00399 P ef41ad9d59be4951885e7143db439d06: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 21:03:34.563992 15147 tablet_bootstrap.cc:492] T 4dc654afcafe4a32bba79b48a707d3c5 P ef41ad9d59be4951885e7143db439d06: No bootstrap required, opened a new log
I20260812 21:03:34.564004 15191 log.cc:1196] T b92996e25ee74e18a6e2cbb46cb00399 P ef41ad9d59be4951885e7143db439d06: Deleting WAL directory at /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568607354921-12282-0/minicluster-data/ts-1/wal/wals/b92996e25ee74e18a6e2cbb46cb00399
I20260812 21:03:34.564071 15147 ts_tablet_manager.cc:1397] T 4dc654afcafe4a32bba79b48a707d3c5 P ef41ad9d59be4951885e7143db439d06: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20260812 21:03:34.564458 15154 tablet_bootstrap.cc:492] T 4dc654afcafe4a32bba79b48a707d3c5 P b715cb5f104640da9111c451705b6d78: No bootstrap required, opened a new log
I20260812 21:03:34.564541 15154 ts_tablet_manager.cc:1397] T 4dc654afcafe4a32bba79b48a707d3c5 P b715cb5f104640da9111c451705b6d78: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.002s
I20260812 21:03:34.564639 15191 ts_tablet_manager.cc:1939] T b92996e25ee74e18a6e2cbb46cb00399 P ef41ad9d59be4951885e7143db439d06: Deleting consensus metadata
I20260812 21:03:34.564944 15147 raft_consensus.cc:348] T 4dc654afcafe4a32bba79b48a707d3c5 P ef41ad9d59be4951885e7143db439d06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e619668bf91d4868bc386cbf6f07c3a2" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 42771 } } peers { permanent_uuid: "ef41ad9d59be4951885e7143db439d06" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 45471 } } peers { permanent_uuid: "b715cb5f104640da9111c451705b6d78" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 41177 } }
I20260812 21:03:34.565090 15147 raft_consensus.cc:374] T 4dc654afcafe4a32bba79b48a707d3c5 P ef41ad9d59be4951885e7143db439d06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:03:34.565129 15147 raft_consensus.cc:729] T 4dc654afcafe4a32bba79b48a707d3c5 P ef41ad9d59be4951885e7143db439d06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef41ad9d59be4951885e7143db439d06, State: Initialized, Role: FOLLOWER
I20260812 21:03:34.565093 15154 raft_consensus.cc:348] T 4dc654afcafe4a32bba79b48a707d3c5 P b715cb5f104640da9111c451705b6d78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e619668bf91d4868bc386cbf6f07c3a2" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 42771 } } peers { permanent_uuid: "ef41ad9d59be4951885e7143db439d06" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 45471 } } peers { permanent_uuid: "b715cb5f104640da9111c451705b6d78" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 41177 } }
I20260812 21:03:34.565187 15154 raft_consensus.cc:374] T 4dc654afcafe4a32bba79b48a707d3c5 P b715cb5f104640da9111c451705b6d78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:03:34.565227 15154 raft_consensus.cc:729] T 4dc654afcafe4a32bba79b48a707d3c5 P b715cb5f104640da9111c451705b6d78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b715cb5f104640da9111c451705b6d78, State: Initialized, Role: FOLLOWER
I20260812 21:03:34.565299 15154 consensus_queue.cc:260] T 4dc654afcafe4a32bba79b48a707d3c5 P b715cb5f104640da9111c451705b6d78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e619668bf91d4868bc386cbf6f07c3a2" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 42771 } } peers { permanent_uuid: "ef41ad9d59be4951885e7143db439d06" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 45471 } } peers { permanent_uuid: "b715cb5f104640da9111c451705b6d78" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 41177 } }
I20260812 21:03:34.565440 15154 ts_tablet_manager.cc:1428] T 4dc654afcafe4a32bba79b48a707d3c5 P b715cb5f104640da9111c451705b6d78: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260812 21:03:34.565712 14223 catalog_manager.cc:4841] TS ef41ad9d59be4951885e7143db439d06 (127.11.254.130:45471): tablet b92996e25ee74e18a6e2cbb46cb00399 (table test-table [id=0b07dad266f34e878498f74c30938e76]) successfully deleted
I20260812 21:03:34.566218 15147 consensus_queue.cc:260] T 4dc654afcafe4a32bba79b48a707d3c5 P ef41ad9d59be4951885e7143db439d06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e619668bf91d4868bc386cbf6f07c3a2" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 42771 } } peers { permanent_uuid: "ef41ad9d59be4951885e7143db439d06" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 45471 } } peers { permanent_uuid: "b715cb5f104640da9111c451705b6d78" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 41177 } }
I20260812 21:03:34.566525 15147 ts_tablet_manager.cc:1428] T 4dc654afcafe4a32bba79b48a707d3c5 P ef41ad9d59be4951885e7143db439d06: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
W20260812 21:03:34.618795 14866 tablet.cc:2369] T 4dc654afcafe4a32bba79b48a707d3c5 P ef41ad9d59be4951885e7143db439d06: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 21:03:34.619130 14623 tablet.cc:2369] T 4dc654afcafe4a32bba79b48a707d3c5 P e619668bf91d4868bc386cbf6f07c3a2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 21:03:34.639008 15111 tablet.cc:2369] T 4dc654afcafe4a32bba79b48a707d3c5 P b715cb5f104640da9111c451705b6d78: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 21:03:34.672876 15159 raft_consensus.cc:482] T 4dc654afcafe4a32bba79b48a707d3c5 P ef41ad9d59be4951885e7143db439d06 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 21:03:34.673018 15159 raft_consensus.cc:504] T 4dc654afcafe4a32bba79b48a707d3c5 P ef41ad9d59be4951885e7143db439d06 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e619668bf91d4868bc386cbf6f07c3a2" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 42771 } } peers { permanent_uuid: "ef41ad9d59be4951885e7143db439d06" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 45471 } } peers { permanent_uuid: "b715cb5f104640da9111c451705b6d78" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 41177 } }
I20260812 21:03:34.673553 15159 leader_election.cc:288] T 4dc654afcafe4a32bba79b48a707d3c5 P ef41ad9d59be4951885e7143db439d06 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e619668bf91d4868bc386cbf6f07c3a2 (127.11.254.129:42771), b715cb5f104640da9111c451705b6d78 (127.11.254.131:41177)
I20260812 21:03:34.678882 15144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:34.674190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.130:56113 (local address 127.11.254.131:41177)
0812 21:03:34.674274 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0812 21:03:34.674275 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 21:03:34.674297 (+ 22us) server_negotiation.cc:407] Connection header received
0812 21:03:34.674342 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:03:34.674343 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:03:34.674358 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:03:34.674471 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:03:34.674667 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:34.675661 (+ 994us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:34.677501 (+ 1840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:34.678120 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:34.678224 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:34.678236 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0812 21:03:34.678717 (+ 481us) server_negotiation.cc:299] Negotiation successful
0812 21:03:34.678788 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260812 21:03:34.679039 15142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:34.673940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.130:50193 (local address 127.11.254.129:42771)
0812 21:03:34.674056 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0812 21:03:34.674058 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:03:34.674216 (+ 158us) server_negotiation.cc:407] Connection header received
0812 21:03:34.674382 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:03:34.674384 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:03:34.674409 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:03:34.674459 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:03:34.674791 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:34.675369 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:34.677449 (+ 2080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:34.678299 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:34.678443 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:34.678463 (+ 20us) server_negotiation.cc:1035] Waiting for connection context
0812 21:03:34.678854 (+ 391us) server_negotiation.cc:299] Negotiation successful
0812 21:03:34.678918 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260812 21:03:34.678956 15203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:34.673859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.131:41177 (local address 127.11.254.130:56113)
0812 21:03:34.674111 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0812 21:03:34.674121 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 21:03:34.674221 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:03:34.674478 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:03:34.674479 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:03:34.674481 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:03:34.674568 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:34.674570 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:34.675782 (+ 1212us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:34.675794 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:34.677300 (+ 1506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:34.677304 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:34.678306 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:34.678308 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:34.678520 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:34.678528 (+ 8us) client_negotiation.cc:769] Sending connection context
0812 21:03:34.678673 (+ 145us) client_negotiation.cc:240] Negotiation successful
0812 21:03:34.678761 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":231,"mutex_wait_us":3,"thread_start_us":137,"threads_started":1}
I20260812 21:03:34.679644 15202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:34.673856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.129:42771 (local address 127.11.254.130:50193)
0812 21:03:34.674088 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0812 21:03:34.674101 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 21:03:34.674251 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:03:34.674549 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:03:34.674550 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:03:34.674552 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:03:34.674699 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:34.674701 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:34.675528 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:34.675530 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:34.677300 (+ 1770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:34.677304 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:34.678536 (+ 1232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:34.678538 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:34.678683 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:34.678693 (+ 10us) client_negotiation.cc:769] Sending connection context
0812 21:03:34.679182 (+ 489us) client_negotiation.cc:240] Negotiation successful
0812 21:03:34.679259 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":207,"mutex_wait_us":2,"thread_start_us":124,"threads_started":1}
I20260812 21:03:34.679824 15032 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4dc654afcafe4a32bba79b48a707d3c5" candidate_uuid: "ef41ad9d59be4951885e7143db439d06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b715cb5f104640da9111c451705b6d78" is_pre_election: true
I20260812 21:03:34.680020 15032 raft_consensus.cc:2454] T 4dc654afcafe4a32bba79b48a707d3c5 P b715cb5f104640da9111c451705b6d78 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef41ad9d59be4951885e7143db439d06 in term 0.
I20260812 21:03:34.680385 14668 leader_election.cc:302] T 4dc654afcafe4a32bba79b48a707d3c5 P ef41ad9d59be4951885e7143db439d06 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b715cb5f104640da9111c451705b6d78, ef41ad9d59be4951885e7143db439d06; no voters:
I20260812 21:03:34.680330 14530 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4dc654afcafe4a32bba79b48a707d3c5" candidate_uuid: "ef41ad9d59be4951885e7143db439d06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e619668bf91d4868bc386cbf6f07c3a2" is_pre_election: true
I20260812 21:03:34.680529 14530 raft_consensus.cc:2454] T 4dc654afcafe4a32bba79b48a707d3c5 P e619668bf91d4868bc386cbf6f07c3a2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef41ad9d59be4951885e7143db439d06 in term 0.
I20260812 21:03:34.680614 15159 raft_consensus.cc:2781] T 4dc654afcafe4a32bba79b48a707d3c5 P ef41ad9d59be4951885e7143db439d06 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 21:03:34.680680 15159 raft_consensus.cc:482] T 4dc654afcafe4a32bba79b48a707d3c5 P ef41ad9d59be4951885e7143db439d06 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 21:03:34.680716 15159 raft_consensus.cc:3037] T 4dc654afcafe4a32bba79b48a707d3c5 P ef41ad9d59be4951885e7143db439d06 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:03:34.681691 15159 raft_consensus.cc:504] T 4dc654afcafe4a32bba79b48a707d3c5 P ef41ad9d59be4951885e7143db439d06 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e619668bf91d4868bc386cbf6f07c3a2" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 42771 } } peers { permanent_uuid: "ef41ad9d59be4951885e7143db439d06" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 45471 } } peers { permanent_uuid: "b715cb5f104640da9111c451705b6d78" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 41177 } }
I20260812 21:03:34.681875 15159 leader_election.cc:288] T 4dc654afcafe4a32bba79b48a707d3c5 P ef41ad9d59be4951885e7143db439d06 [CANDIDATE]: Term 1 election: Requested vote from peers e619668bf91d4868bc386cbf6f07c3a2 (127.11.254.129:42771), b715cb5f104640da9111c451705b6d78 (127.11.254.131:41177)
I20260812 21:03:34.682202 15032 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4dc654afcafe4a32bba79b48a707d3c5" candidate_uuid: "ef41ad9d59be4951885e7143db439d06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b715cb5f104640da9111c451705b6d78"
I20260812 21:03:34.682255 14530 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4dc654afcafe4a32bba79b48a707d3c5" candidate_uuid: "ef41ad9d59be4951885e7143db439d06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e619668bf91d4868bc386cbf6f07c3a2"
I20260812 21:03:34.682338 15032 raft_consensus.cc:3037] T 4dc654afcafe4a32bba79b48a707d3c5 P b715cb5f104640da9111c451705b6d78 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:03:34.682358 14530 raft_consensus.cc:3037] T 4dc654afcafe4a32bba79b48a707d3c5 P e619668bf91d4868bc386cbf6f07c3a2 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:03:34.683641 15032 raft_consensus.cc:2454] T 4dc654afcafe4a32bba79b48a707d3c5 P b715cb5f104640da9111c451705b6d78 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef41ad9d59be4951885e7143db439d06 in term 1.
I20260812 21:03:34.683909 14668 leader_election.cc:302] T 4dc654afcafe4a32bba79b48a707d3c5 P ef41ad9d59be4951885e7143db439d06 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b715cb5f104640da9111c451705b6d78, ef41ad9d59be4951885e7143db439d06; no voters:
I20260812 21:03:34.684063 15159 raft_consensus.cc:2781] T 4dc654afcafe4a32bba79b48a707d3c5 P ef41ad9d59be4951885e7143db439d06 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:03:34.684115 14530 raft_consensus.cc:2454] T 4dc654afcafe4a32bba79b48a707d3c5 P e619668bf91d4868bc386cbf6f07c3a2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef41ad9d59be4951885e7143db439d06 in term 1.
I20260812 21:03:34.684243 15159 raft_consensus.cc:686] T 4dc654afcafe4a32bba79b48a707d3c5 P ef41ad9d59be4951885e7143db439d06 [term 1 LEADER]: Becoming Leader. State: Replica: ef41ad9d59be4951885e7143db439d06, State: Running, Role: LEADER
I20260812 21:03:34.684373 15159 consensus_queue.cc:237] T 4dc654afcafe4a32bba79b48a707d3c5 P ef41ad9d59be4951885e7143db439d06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e619668bf91d4868bc386cbf6f07c3a2" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 42771 } } peers { permanent_uuid: "ef41ad9d59be4951885e7143db439d06" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 45471 } } peers { permanent_uuid: "b715cb5f104640da9111c451705b6d78" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 41177 } }
I20260812 21:03:34.685629 14245 catalog_manager.cc:5495] T 4dc654afcafe4a32bba79b48a707d3c5 P ef41ad9d59be4951885e7143db439d06 reported cstate change: term changed from 0 to 1, leader changed from <none> to ef41ad9d59be4951885e7143db439d06 (127.11.254.130). New cstate: current_term: 1 leader_uuid: "ef41ad9d59be4951885e7143db439d06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e619668bf91d4868bc386cbf6f07c3a2" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 42771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ef41ad9d59be4951885e7143db439d06" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 45471 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b715cb5f104640da9111c451705b6d78" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 41177 } health_report { overall_health: UNKNOWN } } }
Aug 12 21:03:34 dist-test-slave-vpvm krb5kdc[14170](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568613, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.190@KRBTEST.COM for kudu/127.11.254.190@KRBTEST.COM
I20260812 21:03:34.755843 15219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:34.743302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:42795 (local address 127.0.0.1:40156)
0812 21:03:34.743606 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0812 21:03:34.743617 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 21:03:34.743702 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:03:34.743934 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:03:34.743935 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:03:34.744493 (+ 558us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:03:34.744593 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:34.744596 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:34.746012 (+ 1416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:34.746013 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:34.746790 (+ 777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:34.746793 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:34.747514 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:34.747514 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:34.747607 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:34.748404 (+ 797us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:03:34.748410 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:03:34.750662 (+ 2252us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:03:34.753639 (+ 2977us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:03:34.753657 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:03:34.753660 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:03:34.754145 (+ 485us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:03:34.754487 (+ 342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:03:34.754488 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:03:34.754488 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:03:34.754594 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:03:34.755171 (+ 577us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:03:34.755190 (+ 19us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:03:34.755297 (+ 107us) client_negotiation.cc:769] Sending connection context
0812 21:03:34.755447 (+ 150us) client_negotiation.cc:240] Negotiation successful
0812 21:03:34.755627 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":159,"threads_started":1}
I20260812 21:03:34.755972 14332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:34.743391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40156 (local address 127.11.254.190:42795)
0812 21:03:34.743493 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0812 21:03:34.743495 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:03:34.743701 (+ 206us) server_negotiation.cc:407] Connection header received
0812 21:03:34.743771 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:03:34.743773 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:03:34.743786 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:03:34.743934 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:03:34.744726 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:34.745827 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:34.746905 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:34.747336 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:34.747507 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:34.750785 (+ 3278us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:03:34.750787 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:03:34.750787 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:03:34.750805 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:03:34.753427 (+ 2622us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:34.754262 (+ 835us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:34.754264 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:34.754264 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:34.754369 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:34.754711 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:34.754712 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:34.754713 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:34.755047 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:03:34.755179 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0812 21:03:34.755529 (+ 350us) server_negotiation.cc:299] Negotiation successful
0812 21:03:34.755755 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":60,"server-negotiator.queue_time_us":50}
I20260812 21:03:34.760674 14245 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.11.254.190@KRBTEST.COM'} at 127.0.0.1:40156:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260812 21:03:34.761214 14245 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 21:03:34.763669 14992 tablet_service.cc:1467] Processing CreateTablet for tablet 567ed9ef18d04605a50fe65ea20bcf53 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b02077a4077c43ceaededc056bf2899c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 21:03:34.763670 14746 tablet_service.cc:1467] Processing CreateTablet for tablet 567ed9ef18d04605a50fe65ea20bcf53 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b02077a4077c43ceaededc056bf2899c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 21:03:34.763634 14501 tablet_service.cc:1467] Processing CreateTablet for tablet 567ed9ef18d04605a50fe65ea20bcf53 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b02077a4077c43ceaededc056bf2899c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 21:03:34.763967 14746 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 567ed9ef18d04605a50fe65ea20bcf53. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:03:34.763967 14501 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 567ed9ef18d04605a50fe65ea20bcf53. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:03:34.764187 14992 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 567ed9ef18d04605a50fe65ea20bcf53. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:03:34.766916 15147 tablet_bootstrap.cc:492] T 567ed9ef18d04605a50fe65ea20bcf53 P ef41ad9d59be4951885e7143db439d06: Bootstrap starting.
I20260812 21:03:34.767333 15154 tablet_bootstrap.cc:492] T 567ed9ef18d04605a50fe65ea20bcf53 P b715cb5f104640da9111c451705b6d78: Bootstrap starting.
I20260812 21:03:34.768757 15154 tablet_bootstrap.cc:654] T 567ed9ef18d04605a50fe65ea20bcf53 P b715cb5f104640da9111c451705b6d78: Neither blocks nor log segments found. Creating new log.
I20260812 21:03:34.768774 15147 tablet_bootstrap.cc:654] T 567ed9ef18d04605a50fe65ea20bcf53 P ef41ad9d59be4951885e7143db439d06: Neither blocks nor log segments found. Creating new log.
I20260812 21:03:34.770597 15148 tablet_bootstrap.cc:492] T 567ed9ef18d04605a50fe65ea20bcf53 P e619668bf91d4868bc386cbf6f07c3a2: Bootstrap starting.
I20260812 21:03:34.771170 15154 tablet_bootstrap.cc:492] T 567ed9ef18d04605a50fe65ea20bcf53 P b715cb5f104640da9111c451705b6d78: No bootstrap required, opened a new log
I20260812 21:03:34.771255 15154 ts_tablet_manager.cc:1397] T 567ed9ef18d04605a50fe65ea20bcf53 P b715cb5f104640da9111c451705b6d78: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.002s
I20260812 21:03:34.771447 15147 tablet_bootstrap.cc:492] T 567ed9ef18d04605a50fe65ea20bcf53 P ef41ad9d59be4951885e7143db439d06: No bootstrap required, opened a new log
I20260812 21:03:34.771533 15147 ts_tablet_manager.cc:1397] T 567ed9ef18d04605a50fe65ea20bcf53 P ef41ad9d59be4951885e7143db439d06: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20260812 21:03:34.771827 15154 raft_consensus.cc:348] T 567ed9ef18d04605a50fe65ea20bcf53 P b715cb5f104640da9111c451705b6d78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e619668bf91d4868bc386cbf6f07c3a2" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 42771 } } peers { permanent_uuid: "ef41ad9d59be4951885e7143db439d06" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 45471 } } peers { permanent_uuid: "b715cb5f104640da9111c451705b6d78" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 41177 } }
I20260812 21:03:34.771978 15154 raft_consensus.cc:374] T 567ed9ef18d04605a50fe65ea20bcf53 P b715cb5f104640da9111c451705b6d78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:03:34.772002 15154 raft_consensus.cc:729] T 567ed9ef18d04605a50fe65ea20bcf53 P b715cb5f104640da9111c451705b6d78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b715cb5f104640da9111c451705b6d78, State: Initialized, Role: FOLLOWER
I20260812 21:03:34.772130 15148 tablet_bootstrap.cc:654] T 567ed9ef18d04605a50fe65ea20bcf53 P e619668bf91d4868bc386cbf6f07c3a2: Neither blocks nor log segments found. Creating new log.
I20260812 21:03:34.772054 15147 raft_consensus.cc:348] T 567ed9ef18d04605a50fe65ea20bcf53 P ef41ad9d59be4951885e7143db439d06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e619668bf91d4868bc386cbf6f07c3a2" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 42771 } } peers { permanent_uuid: "ef41ad9d59be4951885e7143db439d06" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 45471 } } peers { permanent_uuid: "b715cb5f104640da9111c451705b6d78" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 41177 } }
I20260812 21:03:34.772257 15147 raft_consensus.cc:374] T 567ed9ef18d04605a50fe65ea20bcf53 P ef41ad9d59be4951885e7143db439d06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:03:34.772194 15154 consensus_queue.cc:260] T 567ed9ef18d04605a50fe65ea20bcf53 P b715cb5f104640da9111c451705b6d78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e619668bf91d4868bc386cbf6f07c3a2" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 42771 } } peers { permanent_uuid: "ef41ad9d59be4951885e7143db439d06" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 45471 } } peers { permanent_uuid: "b715cb5f104640da9111c451705b6d78" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 41177 } }
I20260812 21:03:34.772306 15147 raft_consensus.cc:729] T 567ed9ef18d04605a50fe65ea20bcf53 P ef41ad9d59be4951885e7143db439d06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef41ad9d59be4951885e7143db439d06, State: Initialized, Role: FOLLOWER
I20260812 21:03:34.772403 15147 consensus_queue.cc:260] T 567ed9ef18d04605a50fe65ea20bcf53 P ef41ad9d59be4951885e7143db439d06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e619668bf91d4868bc386cbf6f07c3a2" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 42771 } } peers { permanent_uuid: "ef41ad9d59be4951885e7143db439d06" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 45471 } } peers { permanent_uuid: "b715cb5f104640da9111c451705b6d78" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 41177 } }
I20260812 21:03:34.772543 15147 ts_tablet_manager.cc:1428] T 567ed9ef18d04605a50fe65ea20bcf53 P ef41ad9d59be4951885e7143db439d06: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260812 21:03:34.772526 15162 tablet_replica.cc:439] T 567ed9ef18d04605a50fe65ea20bcf53 P b715cb5f104640da9111c451705b6d78: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e619668bf91d4868bc386cbf6f07c3a2" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 42771 } } peers { permanent_uuid: "ef41ad9d59be4951885e7143db439d06" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 45471 } } peers { permanent_uuid: "b715cb5f104640da9111c451705b6d78" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 41177 } } }
I20260812 21:03:34.772797 15162 tablet_replica.cc:442] T 567ed9ef18d04605a50fe65ea20bcf53 P b715cb5f104640da9111c451705b6d78: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 21:03:34.772957 15159 tablet_replica.cc:439] T 567ed9ef18d04605a50fe65ea20bcf53 P ef41ad9d59be4951885e7143db439d06: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e619668bf91d4868bc386cbf6f07c3a2" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 42771 } } peers { permanent_uuid: "ef41ad9d59be4951885e7143db439d06" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 45471 } } peers { permanent_uuid: "b715cb5f104640da9111c451705b6d78" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 41177 } } }
I20260812 21:03:34.773113 15159 tablet_replica.cc:442] T 567ed9ef18d04605a50fe65ea20bcf53 P ef41ad9d59be4951885e7143db439d06: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 21:03:34.773196 15154 ts_tablet_manager.cc:1428] T 567ed9ef18d04605a50fe65ea20bcf53 P b715cb5f104640da9111c451705b6d78: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260812 21:03:34.774250 15148 tablet_bootstrap.cc:492] T 567ed9ef18d04605a50fe65ea20bcf53 P e619668bf91d4868bc386cbf6f07c3a2: No bootstrap required, opened a new log
I20260812 21:03:34.774391 15148 ts_tablet_manager.cc:1397] T 567ed9ef18d04605a50fe65ea20bcf53 P e619668bf91d4868bc386cbf6f07c3a2: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.003s
I20260812 21:03:34.774992 15148 raft_consensus.cc:348] T 567ed9ef18d04605a50fe65ea20bcf53 P e619668bf91d4868bc386cbf6f07c3a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e619668bf91d4868bc386cbf6f07c3a2" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 42771 } } peers { permanent_uuid: "ef41ad9d59be4951885e7143db439d06" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 45471 } } peers { permanent_uuid: "b715cb5f104640da9111c451705b6d78" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 41177 } }
I20260812 21:03:34.775238 15148 raft_consensus.cc:374] T 567ed9ef18d04605a50fe65ea20bcf53 P e619668bf91d4868bc386cbf6f07c3a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:03:34.775292 15148 raft_consensus.cc:729] T 567ed9ef18d04605a50fe65ea20bcf53 P e619668bf91d4868bc386cbf6f07c3a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e619668bf91d4868bc386cbf6f07c3a2, State: Initialized, Role: FOLLOWER
I20260812 21:03:34.775416 15148 consensus_queue.cc:260] T 567ed9ef18d04605a50fe65ea20bcf53 P e619668bf91d4868bc386cbf6f07c3a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e619668bf91d4868bc386cbf6f07c3a2" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 42771 } } peers { permanent_uuid: "ef41ad9d59be4951885e7143db439d06" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 45471 } } peers { permanent_uuid: "b715cb5f104640da9111c451705b6d78" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 41177 } }
I20260812 21:03:34.775575 15148 ts_tablet_manager.cc:1428] T 567ed9ef18d04605a50fe65ea20bcf53 P e619668bf91d4868bc386cbf6f07c3a2: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 21:03:34.775615 15160 tablet_replica.cc:439] T 567ed9ef18d04605a50fe65ea20bcf53 P e619668bf91d4868bc386cbf6f07c3a2: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e619668bf91d4868bc386cbf6f07c3a2" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 42771 } } peers { permanent_uuid: "ef41ad9d59be4951885e7143db439d06" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 45471 } } peers { permanent_uuid: "b715cb5f104640da9111c451705b6d78" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 41177 } } }
I20260812 21:03:34.775770 15160 tablet_replica.cc:442] T 567ed9ef18d04605a50fe65ea20bcf53 P e619668bf91d4868bc386cbf6f07c3a2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 21:03:35.110528 15159 consensus_queue.cc:1035] T 4dc654afcafe4a32bba79b48a707d3c5 P ef41ad9d59be4951885e7143db439d06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e619668bf91d4868bc386cbf6f07c3a2" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 42771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 21:03:35.113731 15230 consensus_queue.cc:1035] T 4dc654afcafe4a32bba79b48a707d3c5 P ef41ad9d59be4951885e7143db439d06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b715cb5f104640da9111c451705b6d78" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 41177 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 21:03:35.132450 15230 raft_consensus.cc:482] T 567ed9ef18d04605a50fe65ea20bcf53 P ef41ad9d59be4951885e7143db439d06 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 21:03:35.132661 15230 raft_consensus.cc:504] T 567ed9ef18d04605a50fe65ea20bcf53 P ef41ad9d59be4951885e7143db439d06 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e619668bf91d4868bc386cbf6f07c3a2" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 42771 } } peers { permanent_uuid: "ef41ad9d59be4951885e7143db439d06" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 45471 } } peers { permanent_uuid: "b715cb5f104640da9111c451705b6d78" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 41177 } }
I20260812 21:03:35.132934 15230 leader_election.cc:288] T 567ed9ef18d04605a50fe65ea20bcf53 P ef41ad9d59be4951885e7143db439d06 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e619668bf91d4868bc386cbf6f07c3a2 (127.11.254.129:42771), b715cb5f104640da9111c451705b6d78 (127.11.254.131:41177)
I20260812 21:03:35.133349 15032 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "567ed9ef18d04605a50fe65ea20bcf53" candidate_uuid: "ef41ad9d59be4951885e7143db439d06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b715cb5f104640da9111c451705b6d78" is_pre_election: true
I20260812 21:03:35.133343 14530 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "567ed9ef18d04605a50fe65ea20bcf53" candidate_uuid: "ef41ad9d59be4951885e7143db439d06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e619668bf91d4868bc386cbf6f07c3a2" is_pre_election: true
I20260812 21:03:35.133525 15032 raft_consensus.cc:2454] T 567ed9ef18d04605a50fe65ea20bcf53 P b715cb5f104640da9111c451705b6d78 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef41ad9d59be4951885e7143db439d06 in term 0.
I20260812 21:03:35.133525 14530 raft_consensus.cc:2454] T 567ed9ef18d04605a50fe65ea20bcf53 P e619668bf91d4868bc386cbf6f07c3a2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef41ad9d59be4951885e7143db439d06 in term 0.
I20260812 21:03:35.133783 14668 leader_election.cc:302] T 567ed9ef18d04605a50fe65ea20bcf53 P ef41ad9d59be4951885e7143db439d06 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b715cb5f104640da9111c451705b6d78, ef41ad9d59be4951885e7143db439d06; no voters:
I20260812 21:03:35.134011 15230 raft_consensus.cc:2781] T 567ed9ef18d04605a50fe65ea20bcf53 P ef41ad9d59be4951885e7143db439d06 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 21:03:35.134092 15230 raft_consensus.cc:482] T 567ed9ef18d04605a50fe65ea20bcf53 P ef41ad9d59be4951885e7143db439d06 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 21:03:35.134140 15230 raft_consensus.cc:3037] T 567ed9ef18d04605a50fe65ea20bcf53 P ef41ad9d59be4951885e7143db439d06 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:03:35.135229 15230 raft_consensus.cc:504] T 567ed9ef18d04605a50fe65ea20bcf53 P ef41ad9d59be4951885e7143db439d06 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e619668bf91d4868bc386cbf6f07c3a2" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 42771 } } peers { permanent_uuid: "ef41ad9d59be4951885e7143db439d06" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 45471 } } peers { permanent_uuid: "b715cb5f104640da9111c451705b6d78" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 41177 } }
I20260812 21:03:35.135439 15230 leader_election.cc:288] T 567ed9ef18d04605a50fe65ea20bcf53 P ef41ad9d59be4951885e7143db439d06 [CANDIDATE]: Term 1 election: Requested vote from peers e619668bf91d4868bc386cbf6f07c3a2 (127.11.254.129:42771), b715cb5f104640da9111c451705b6d78 (127.11.254.131:41177)
I20260812 21:03:35.135751 15032 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "567ed9ef18d04605a50fe65ea20bcf53" candidate_uuid: "ef41ad9d59be4951885e7143db439d06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b715cb5f104640da9111c451705b6d78"
I20260812 21:03:35.135779 14530 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "567ed9ef18d04605a50fe65ea20bcf53" candidate_uuid: "ef41ad9d59be4951885e7143db439d06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e619668bf91d4868bc386cbf6f07c3a2"
I20260812 21:03:35.135895 15032 raft_consensus.cc:3037] T 567ed9ef18d04605a50fe65ea20bcf53 P b715cb5f104640da9111c451705b6d78 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:03:35.135895 14530 raft_consensus.cc:3037] T 567ed9ef18d04605a50fe65ea20bcf53 P e619668bf91d4868bc386cbf6f07c3a2 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:03:35.137138 15032 raft_consensus.cc:2454] T 567ed9ef18d04605a50fe65ea20bcf53 P b715cb5f104640da9111c451705b6d78 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef41ad9d59be4951885e7143db439d06 in term 1.
I20260812 21:03:35.137156 14530 raft_consensus.cc:2454] T 567ed9ef18d04605a50fe65ea20bcf53 P e619668bf91d4868bc386cbf6f07c3a2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef41ad9d59be4951885e7143db439d06 in term 1.
I20260812 21:03:35.137470 14671 leader_election.cc:302] T 567ed9ef18d04605a50fe65ea20bcf53 P ef41ad9d59be4951885e7143db439d06 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e619668bf91d4868bc386cbf6f07c3a2, ef41ad9d59be4951885e7143db439d06; no voters:
I20260812 21:03:35.137630 15230 raft_consensus.cc:2781] T 567ed9ef18d04605a50fe65ea20bcf53 P ef41ad9d59be4951885e7143db439d06 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:03:35.137714 15230 raft_consensus.cc:686] T 567ed9ef18d04605a50fe65ea20bcf53 P ef41ad9d59be4951885e7143db439d06 [term 1 LEADER]: Becoming Leader. State: Replica: ef41ad9d59be4951885e7143db439d06, State: Running, Role: LEADER
I20260812 21:03:35.137806 15230 consensus_queue.cc:237] T 567ed9ef18d04605a50fe65ea20bcf53 P ef41ad9d59be4951885e7143db439d06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e619668bf91d4868bc386cbf6f07c3a2" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 42771 } } peers { permanent_uuid: "ef41ad9d59be4951885e7143db439d06" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 45471 } } peers { permanent_uuid: "b715cb5f104640da9111c451705b6d78" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 41177 } }
I20260812 21:03:35.138273 15159 tablet_replica.cc:439] T 567ed9ef18d04605a50fe65ea20bcf53 P ef41ad9d59be4951885e7143db439d06: TxnStatusTablet state changed. Reason: New leader ef41ad9d59be4951885e7143db439d06. Latest consensus state: current_term: 1 leader_uuid: "ef41ad9d59be4951885e7143db439d06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e619668bf91d4868bc386cbf6f07c3a2" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 42771 } } peers { permanent_uuid: "ef41ad9d59be4951885e7143db439d06" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 45471 } } peers { permanent_uuid: "b715cb5f104640da9111c451705b6d78" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 41177 } } }
I20260812 21:03:35.138381 15159 tablet_replica.cc:442] T 567ed9ef18d04605a50fe65ea20bcf53 P ef41ad9d59be4951885e7143db439d06: This TxnStatusTablet replica's current role is: LEADER
I20260812 21:03:35.138604 15236 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260812 21:03:35.138886 15236 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260812 21:03:35.138886 14246 catalog_manager.cc:5495] T 567ed9ef18d04605a50fe65ea20bcf53 P ef41ad9d59be4951885e7143db439d06 reported cstate change: term changed from 0 to 1, leader changed from <none> to ef41ad9d59be4951885e7143db439d06 (127.11.254.130). New cstate: current_term: 1 leader_uuid: "ef41ad9d59be4951885e7143db439d06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e619668bf91d4868bc386cbf6f07c3a2" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 42771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ef41ad9d59be4951885e7143db439d06" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 45471 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b715cb5f104640da9111c451705b6d78" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 41177 } health_report { overall_health: UNKNOWN } } }
I20260812 21:03:35.223204 15219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:35.217347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.130:45471 (local address 127.0.0.1:43422)
0812 21:03:35.217469 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0812 21:03:35.217492 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0812 21:03:35.217592 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:03:35.218094 (+ 502us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:03:35.218096 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:03:35.218097 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 21:03:35.218204 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:35.218207 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:35.220511 (+ 2304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:35.220513 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:35.221464 (+ 951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:35.221468 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:35.222234 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:35.222235 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:35.222330 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:35.222363 (+ 33us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 21:03:35.222867 (+ 504us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 21:03:35.222882 (+ 15us) client_negotiation.cc:769] Sending connection context
0812 21:03:35.222978 (+ 96us) client_negotiation.cc:240] Negotiation successful
0812 21:03:35.223076 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260812 21:03:35.223271 15242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:35.217633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43422 (local address 127.11.254.130:45471)
0812 21:03:35.217823 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0812 21:03:35.217825 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:03:35.217838 (+ 13us) server_negotiation.cc:407] Connection header received
0812 21:03:35.217874 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:03:35.217875 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:03:35.217909 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:03:35.218090 (+ 181us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 21:03:35.218292 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:35.220200 (+ 1908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:35.221593 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:35.222117 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:35.222230 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:35.222617 (+ 387us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 21:03:35.222776 (+ 159us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 21:03:35.222884 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0812 21:03:35.222999 (+ 115us) server_negotiation.cc:299] Negotiation successful
0812 21:03:35.223047 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":101,"threads_started":1}
I20260812 21:03:35.225432 15032 raft_consensus.cc:1261] T 567ed9ef18d04605a50fe65ea20bcf53 P b715cb5f104640da9111c451705b6d78 [term 1 FOLLOWER]: Refusing update from remote peer ef41ad9d59be4951885e7143db439d06: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 21:03:35.225592 14530 raft_consensus.cc:1261] T 567ed9ef18d04605a50fe65ea20bcf53 P e619668bf91d4868bc386cbf6f07c3a2 [term 1 FOLLOWER]: Refusing update from remote peer ef41ad9d59be4951885e7143db439d06: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 21:03:35.225986 15159 consensus_queue.cc:1035] T 567ed9ef18d04605a50fe65ea20bcf53 P ef41ad9d59be4951885e7143db439d06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b715cb5f104640da9111c451705b6d78" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 41177 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 21:03:35.226209 15230 consensus_queue.cc:1035] T 567ed9ef18d04605a50fe65ea20bcf53 P ef41ad9d59be4951885e7143db439d06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e619668bf91d4868bc386cbf6f07c3a2" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 42771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 21:03:35.227068 15162 tablet_replica.cc:439] T 567ed9ef18d04605a50fe65ea20bcf53 P b715cb5f104640da9111c451705b6d78: TxnStatusTablet state changed. Reason: New leader ef41ad9d59be4951885e7143db439d06. Latest consensus state: current_term: 1 leader_uuid: "ef41ad9d59be4951885e7143db439d06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e619668bf91d4868bc386cbf6f07c3a2" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 42771 } } peers { permanent_uuid: "ef41ad9d59be4951885e7143db439d06" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 45471 } } peers { permanent_uuid: "b715cb5f104640da9111c451705b6d78" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 41177 } } }
I20260812 21:03:35.227234 15162 tablet_replica.cc:442] T 567ed9ef18d04605a50fe65ea20bcf53 P b715cb5f104640da9111c451705b6d78: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 21:03:35.227454 15160 tablet_replica.cc:439] T 567ed9ef18d04605a50fe65ea20bcf53 P e619668bf91d4868bc386cbf6f07c3a2: TxnStatusTablet state changed. Reason: New leader ef41ad9d59be4951885e7143db439d06. Latest consensus state: current_term: 1 leader_uuid: "ef41ad9d59be4951885e7143db439d06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e619668bf91d4868bc386cbf6f07c3a2" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 42771 } } peers { permanent_uuid: "ef41ad9d59be4951885e7143db439d06" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 45471 } } peers { permanent_uuid: "b715cb5f104640da9111c451705b6d78" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 41177 } } }
I20260812 21:03:35.227602 15160 tablet_replica.cc:442] T 567ed9ef18d04605a50fe65ea20bcf53 P e619668bf91d4868bc386cbf6f07c3a2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 21:03:35.229075 15204 tablet_replica.cc:439] T 567ed9ef18d04605a50fe65ea20bcf53 P ef41ad9d59be4951885e7143db439d06: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ef41ad9d59be4951885e7143db439d06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e619668bf91d4868bc386cbf6f07c3a2" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 42771 } } peers { permanent_uuid: "ef41ad9d59be4951885e7143db439d06" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 45471 } } peers { permanent_uuid: "b715cb5f104640da9111c451705b6d78" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 41177 } } }
I20260812 21:03:35.229169 15159 tablet_replica.cc:439] T 567ed9ef18d04605a50fe65ea20bcf53 P ef41ad9d59be4951885e7143db439d06: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ef41ad9d59be4951885e7143db439d06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e619668bf91d4868bc386cbf6f07c3a2" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 42771 } } peers { permanent_uuid: "ef41ad9d59be4951885e7143db439d06" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 45471 } } peers { permanent_uuid: "b715cb5f104640da9111c451705b6d78" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 41177 } } }
I20260812 21:03:35.229293 15204 tablet_replica.cc:442] T 567ed9ef18d04605a50fe65ea20bcf53 P ef41ad9d59be4951885e7143db439d06: This TxnStatusTablet replica's current role is: LEADER
I20260812 21:03:35.229300 15159 tablet_replica.cc:442] T 567ed9ef18d04605a50fe65ea20bcf53 P ef41ad9d59be4951885e7143db439d06: This TxnStatusTablet replica's current role is: LEADER
I20260812 21:03:35.229503 15162 tablet_replica.cc:439] T 567ed9ef18d04605a50fe65ea20bcf53 P b715cb5f104640da9111c451705b6d78: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ef41ad9d59be4951885e7143db439d06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e619668bf91d4868bc386cbf6f07c3a2" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 42771 } } peers { permanent_uuid: "ef41ad9d59be4951885e7143db439d06" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 45471 } } peers { permanent_uuid: "b715cb5f104640da9111c451705b6d78" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 41177 } } }
I20260812 21:03:35.229661 15162 tablet_replica.cc:442] T 567ed9ef18d04605a50fe65ea20bcf53 P b715cb5f104640da9111c451705b6d78: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 21:03:35.229753 15227 mvcc.cc:205] Tried to move back new op lower bound from 7317785047960743936 to 7317785047605428224. Current Snapshot: MvccSnapshot[applied={T|T < 7317785047960743936}]
I20260812 21:03:35.230533 15160 tablet_replica.cc:439] T 567ed9ef18d04605a50fe65ea20bcf53 P e619668bf91d4868bc386cbf6f07c3a2: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ef41ad9d59be4951885e7143db439d06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e619668bf91d4868bc386cbf6f07c3a2" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 42771 } } peers { permanent_uuid: "ef41ad9d59be4951885e7143db439d06" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 45471 } } peers { permanent_uuid: "b715cb5f104640da9111c451705b6d78" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 41177 } } }
I20260812 21:03:35.230692 15160 tablet_replica.cc:442] T 567ed9ef18d04605a50fe65ea20bcf53 P e619668bf91d4868bc386cbf6f07c3a2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 21:03:35.237892 15242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:35.233458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43438 (local address 127.11.254.130:45471)
0812 21:03:35.233528 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0812 21:03:35.233530 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:03:35.233680 (+ 150us) server_negotiation.cc:407] Connection header received
0812 21:03:35.233813 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:03:35.233814 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:03:35.233830 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:03:35.233908 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 21:03:35.234165 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:35.235103 (+ 938us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:35.236873 (+ 1770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:35.237172 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:35.237257 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:35.237537 (+ 280us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 21:03:35.237581 (+ 44us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 21:03:35.237657 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 21:03:35.237774 (+ 117us) server_negotiation.cc:299] Negotiation successful
0812 21:03:35.237811 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260812 21:03:35.248807 15242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:35.243211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.130:39417 (local address 127.11.254.130:45471)
0812 21:03:35.243306 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0812 21:03:35.243309 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 21:03:35.243777 (+ 468us) server_negotiation.cc:407] Connection header received
0812 21:03:35.243919 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:03:35.243920 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:03:35.243930 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:03:35.244002 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 21:03:35.244426 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:35.245220 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:35.247314 (+ 2094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:35.247583 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:35.247653 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:35.248347 (+ 694us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 21:03:35.248402 (+ 55us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 21:03:35.248463 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0812 21:03:35.248622 (+ 159us) server_negotiation.cc:299] Negotiation successful
0812 21:03:35.248661 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260812 21:03:35.249024 15257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:35.243098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.130:45471 (local address 127.11.254.130:39417)
0812 21:03:35.243658 (+ 560us) negotiation.cc:107] Waiting for socket to connect
0812 21:03:35.243678 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 21:03:35.243797 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:03:35.244008 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:03:35.244008 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:03:35.244020 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 21:03:35.244296 (+ 276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:35.244305 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:35.245346 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:35.245347 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:35.247156 (+ 1809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:35.247173 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:35.247780 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:35.247781 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:35.247986 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:35.248025 (+ 39us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 21:03:35.248550 (+ 525us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 21:03:35.248552 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 21:03:35.248610 (+ 58us) client_negotiation.cc:240] Negotiation successful
0812 21:03:35.248686 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":255,"threads_started":1}
I20260812 21:03:35.272326 14246 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40146:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 21:03:35.272491 14246 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40146:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 21:03:35.273667 14246 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f45b7349c97e46c59225cf0ba1c645e0: Sending DeleteTablet for 3 replicas of tablet 4dc654afcafe4a32bba79b48a707d3c5
I20260812 21:03:35.274096 14992 tablet_service.cc:1514] Processing DeleteTablet for tablet 4dc654afcafe4a32bba79b48a707d3c5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:03:35 UTC) from {username='kudu', principal='kudu/127.11.254.190@KRBTEST.COM'} at 127.0.0.1:46312
I20260812 21:03:35.274330 14746 tablet_service.cc:1514] Processing DeleteTablet for tablet 4dc654afcafe4a32bba79b48a707d3c5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:03:35 UTC) from {username='kudu', principal='kudu/127.11.254.190@KRBTEST.COM'} at 127.0.0.1:43420
I20260812 21:03:35.274724 15265 tablet_replica.cc:330] T 4dc654afcafe4a32bba79b48a707d3c5 P ef41ad9d59be4951885e7143db439d06: stopping tablet replica
I20260812 21:03:35.274868 15265 raft_consensus.cc:2229] T 4dc654afcafe4a32bba79b48a707d3c5 P ef41ad9d59be4951885e7143db439d06 [term 1 LEADER]: Raft consensus shutting down.
I20260812 21:03:35.275045 15265 raft_consensus.cc:2258] T 4dc654afcafe4a32bba79b48a707d3c5 P ef41ad9d59be4951885e7143db439d06 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 21:03:35.275359 14501 tablet_service.cc:1514] Processing DeleteTablet for tablet 4dc654afcafe4a32bba79b48a707d3c5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:03:35 UTC) from {username='kudu', principal='kudu/127.11.254.190@KRBTEST.COM'} at 127.0.0.1:37766
I20260812 21:03:35.275538 15265 ts_tablet_manager.cc:1905] T 4dc654afcafe4a32bba79b48a707d3c5 P ef41ad9d59be4951885e7143db439d06: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 21:03:35.275774 15264 tablet_replica.cc:330] T 4dc654afcafe4a32bba79b48a707d3c5 P b715cb5f104640da9111c451705b6d78: stopping tablet replica
I20260812 21:03:35.275879 15264 raft_consensus.cc:2229] T 4dc654afcafe4a32bba79b48a707d3c5 P b715cb5f104640da9111c451705b6d78 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 21:03:35.275959 15264 raft_consensus.cc:2258] T 4dc654afcafe4a32bba79b48a707d3c5 P b715cb5f104640da9111c451705b6d78 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 21:03:35.276127 15267 tablet_replica.cc:330] T 4dc654afcafe4a32bba79b48a707d3c5 P e619668bf91d4868bc386cbf6f07c3a2: stopping tablet replica
I20260812 21:03:35.276224 15267 raft_consensus.cc:2229] T 4dc654afcafe4a32bba79b48a707d3c5 P e619668bf91d4868bc386cbf6f07c3a2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 21:03:35.276306 15267 raft_consensus.cc:2258] T 4dc654afcafe4a32bba79b48a707d3c5 P e619668bf91d4868bc386cbf6f07c3a2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 21:03:35.276600 15264 ts_tablet_manager.cc:1905] T 4dc654afcafe4a32bba79b48a707d3c5 P b715cb5f104640da9111c451705b6d78: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 21:03:35.276749 15267 ts_tablet_manager.cc:1905] T 4dc654afcafe4a32bba79b48a707d3c5 P e619668bf91d4868bc386cbf6f07c3a2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 21:03:35.278467 15265 ts_tablet_manager.cc:1918] T 4dc654afcafe4a32bba79b48a707d3c5 P ef41ad9d59be4951885e7143db439d06: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 21:03:35.278584 15265 log.cc:1196] T 4dc654afcafe4a32bba79b48a707d3c5 P ef41ad9d59be4951885e7143db439d06: Deleting WAL directory at /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568607354921-12282-0/minicluster-data/ts-1/wal/wals/4dc654afcafe4a32bba79b48a707d3c5
I20260812 21:03:35.279039 15265 ts_tablet_manager.cc:1939] T 4dc654afcafe4a32bba79b48a707d3c5 P ef41ad9d59be4951885e7143db439d06: Deleting consensus metadata
I20260812 21:03:35.279568 14223 catalog_manager.cc:4841] TS ef41ad9d59be4951885e7143db439d06 (127.11.254.130:45471): tablet 4dc654afcafe4a32bba79b48a707d3c5 (table test-table [id=bd60207299a94d0cb877232316eab99b]) successfully deleted
I20260812 21:03:35.279842 15267 ts_tablet_manager.cc:1918] T 4dc654afcafe4a32bba79b48a707d3c5 P e619668bf91d4868bc386cbf6f07c3a2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 21:03:35.279975 15267 log.cc:1196] T 4dc654afcafe4a32bba79b48a707d3c5 P e619668bf91d4868bc386cbf6f07c3a2: Deleting WAL directory at /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568607354921-12282-0/minicluster-data/ts-0/wal/wals/4dc654afcafe4a32bba79b48a707d3c5
I20260812 21:03:35.280050 15264 ts_tablet_manager.cc:1918] T 4dc654afcafe4a32bba79b48a707d3c5 P b715cb5f104640da9111c451705b6d78: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 21:03:35.280156 15264 log.cc:1196] T 4dc654afcafe4a32bba79b48a707d3c5 P b715cb5f104640da9111c451705b6d78: Deleting WAL directory at /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786568607354921-12282-0/minicluster-data/ts-2/wal/wals/4dc654afcafe4a32bba79b48a707d3c5
I20260812 21:03:35.280404 15267 ts_tablet_manager.cc:1939] T 4dc654afcafe4a32bba79b48a707d3c5 P e619668bf91d4868bc386cbf6f07c3a2: Deleting consensus metadata
I20260812 21:03:35.280524 15264 ts_tablet_manager.cc:1939] T 4dc654afcafe4a32bba79b48a707d3c5 P b715cb5f104640da9111c451705b6d78: Deleting consensus metadata
I20260812 21:03:35.280911 14221 catalog_manager.cc:4841] TS e619668bf91d4868bc386cbf6f07c3a2 (127.11.254.129:42771): tablet 4dc654afcafe4a32bba79b48a707d3c5 (table test-table [id=bd60207299a94d0cb877232316eab99b]) successfully deleted
I20260812 21:03:35.280961 14219 catalog_manager.cc:4841] TS b715cb5f104640da9111c451705b6d78 (127.11.254.131:41177): tablet 4dc654afcafe4a32bba79b48a707d3c5 (table test-table [id=bd60207299a94d0cb877232316eab99b]) successfully deleted
Aug 12 21:03:35 dist-test-slave-vpvm krb5kdc[14170](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568614, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.11.254.129@KRBTEST.COM
I20260812 21:03:35.296285 15268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:35.275605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37788 (local address 127.11.254.129:42771)
0812 21:03:35.279835 (+ 4230us) server_negotiation.cc:206] Beginning negotiation
0812 21:03:35.279838 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 21:03:35.282260 (+ 2422us) server_negotiation.cc:407] Connection header received
0812 21:03:35.282370 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:03:35.282371 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:03:35.282447 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:03:35.282572 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:03:35.283833 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:35.284522 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:35.286374 (+ 1852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:35.286830 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:35.286961 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:35.291440 (+ 4479us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:03:35.291462 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:03:35.291462 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:03:35.291483 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:03:35.294513 (+ 3030us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:35.295033 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:35.295037 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:35.295038 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:35.295120 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:35.295405 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:35.295407 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:35.295407 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:35.295639 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:03:35.295854 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0812 21:03:35.295977 (+ 123us) server_negotiation.cc:299] Negotiation successful
0812 21:03:35.296112 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4188,"thread_start_us":83,"threads_started":1}
W20260812 21:03:35.296653 14609 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37788
I20260812 21:03:35.308627 15277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:35.298238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40162 (local address 127.11.254.190:42795)
0812 21:03:35.298393 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0812 21:03:35.298395 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:03:35.298614 (+ 219us) server_negotiation.cc:407] Connection header received
0812 21:03:35.298742 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:03:35.298743 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:03:35.298766 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:03:35.298859 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:03:35.299835 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:35.300544 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:35.301561 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:35.301838 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:35.301921 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:35.303870 (+ 1949us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:03:35.303871 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:03:35.303872 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:03:35.303890 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:03:35.306802 (+ 2912us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:35.307414 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:35.307415 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:35.307416 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:35.307516 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:35.307773 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:35.307774 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:35.307774 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:35.307960 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:03:35.308100 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0812 21:03:35.308272 (+ 172us) server_negotiation.cc:299] Negotiation successful
0812 21:03:35.308409 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":76,"threads_started":1}
W20260812 21:03:35.309207 14246 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40162
I20260812 21:03:35.310802 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 14371
I20260812 21:03:35.333300 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 14646
I20260812 21:03:35.352967 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 14881
I20260812 21:03:35.373427 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 14193
2026-08-12T21:03:35Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (3293 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568607354921-12282-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 21:03:35 dist-test-slave-vpvm krb5kdc[15282](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 21:03:35 dist-test-slave-vpvm krb5kdc[15282](info): set up 2 sockets
Aug 12 21:03:35 dist-test-slave-vpvm krb5kdc[15282](info): commencing operation
krb5kdc: starting...
W20260812 21:03:36.641892 12282 mini_kdc.cc:121] Time spent starting KDC: real 1.232s user 0.004s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568607354921-12282-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568607354921-12282-0/krb5kdc/test-user.keytab.
Aug 12 21:03:36 dist-test-slave-vpvm krb5kdc[15282](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568616, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T21:03:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T21:03:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.254.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.254.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.254.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.190.keytab.
Entry for principal kudu/127.11.254.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.190.keytab.
I20260812 21:03:36.765264 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568607354921-12282-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568607354921-12282-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568607354921-12282-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.254.190:45761
--webserver_interface=127.11.254.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.190.keytab
--principal=kudu/127.11.254.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.254.148:37257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.254.190:45761
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568607354921-12282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568607354921-12282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568607354921-12282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:03:36.847522 15301 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:03:36.847724 15301 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:03:36.847745 15301 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:03:36.848873 15301 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 21:03:36.848914 15301 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260812 21:03:36.848927 15301 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:03:36.848938 15301 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:03:36.848953 15301 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 21:03:36.848982 15301 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 21:03:36.850520 15301 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.254.148:37257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568607354921-12282-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568607354921-12282-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.254.190:45761
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.190.keytab
--principal=kudu/127.11.254.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.254.190:45761
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.254.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15301
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568607354921-12282-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:03:36.850741 15301 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:03:36.850975 15301 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:03:36.855692 15320 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:03:36.855749 15313 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:03:36.855880 15315 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:03:36.855954 15301 server_base.cc:1034] running on GCE node
I20260812 21:03:36.856175 15301 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:03:36.856451 15301 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:03:36.857604 15301 hybrid_clock.cc:648] HybridClock initialized: now 1786568616857579 us; error 36 us; skew 500 ppm
Aug 12 21:03:36 dist-test-slave-vpvm krb5kdc[15282](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568616, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:03:36.860220 15301 init.cc:377] Logged in from keytab as kudu/127.11.254.190@KRBTEST.COM (short username kudu)
I20260812 21:03:36.860641 15301 webserver.cc:460] Webserver started at http://127.11.254.190:44037/ using document root <none> and password file <none>
I20260812 21:03:36.860843 15301 fs_manager.cc:359] Metadata directory not provided
I20260812 21:03:36.860882 15301 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:03:36.860980 15301 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:03:36.862040 15301 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568607354921-12282-0/minicluster-data/master-0/data/instance:
uuid: "f41525e3e1134a4887fe0909de2f21f6"
format_stamp: "Formatted at 2026-08-12 21:03:36 on dist-test-slave-vpvm"
server_key: "4597eeb0488e6d06e213aaaee19c30a8"
server_key_iv: "a371915243ffd7f5b16daf71fe51e89d"
server_key_version: "encryptionkey@0"
I20260812 21:03:36.862370 15301 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568607354921-12282-0/minicluster-data/master-0/wal/instance:
uuid: "f41525e3e1134a4887fe0909de2f21f6"
format_stamp: "Formatted at 2026-08-12 21:03:36 on dist-test-slave-vpvm"
server_key: "4597eeb0488e6d06e213aaaee19c30a8"
server_key_iv: "a371915243ffd7f5b16daf71fe51e89d"
server_key_version: "encryptionkey@0"
I20260812 21:03:36.863682 15301 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 21:03:36.864424 15333 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:03:36.864589 15301 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 21:03:36.864677 15301 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568607354921-12282-0/minicluster-data/master-0/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568607354921-12282-0/minicluster-data/master-0/wal
uuid: "f41525e3e1134a4887fe0909de2f21f6"
format_stamp: "Formatted at 2026-08-12 21:03:36 on dist-test-slave-vpvm"
server_key: "4597eeb0488e6d06e213aaaee19c30a8"
server_key_iv: "a371915243ffd7f5b16daf71fe51e89d"
server_key_version: "encryptionkey@0"
I20260812 21:03:36.864740 15301 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568607354921-12282-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568607354921-12282-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568607354921-12282-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:03:36.894529 15301 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:03:36.896610 15301 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:03:36.896756 15301 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:03:36.901278 15301 rpc_server.cc:307] RPC server started. Bound to: 127.11.254.190:45761
I20260812 21:03:36.901364 15456 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.254.190:45761 every 8 connection(s)
I20260812 21:03:36.901662 15301 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
I20260812 21:03:36.902331 15458 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:03:36.904703 15458 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f41525e3e1134a4887fe0909de2f21f6: Bootstrap starting.
I20260812 21:03:36.905371 15458 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f41525e3e1134a4887fe0909de2f21f6: Neither blocks nor log segments found. Creating new log.
I20260812 21:03:36.905658 15458 log.cc:824] T 00000000000000000000000000000000 P f41525e3e1134a4887fe0909de2f21f6: Log is configured to *not* fsync() on all Append() calls
I20260812 21:03:36.906347 15458 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f41525e3e1134a4887fe0909de2f21f6: No bootstrap required, opened a new log
I20260812 21:03:36.908604 15458 raft_consensus.cc:348] T 00000000000000000000000000000000 P f41525e3e1134a4887fe0909de2f21f6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f41525e3e1134a4887fe0909de2f21f6" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 45761 } }
I20260812 21:03:36.908758 15458 raft_consensus.cc:374] T 00000000000000000000000000000000 P f41525e3e1134a4887fe0909de2f21f6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:03:36.908792 15458 raft_consensus.cc:729] T 00000000000000000000000000000000 P f41525e3e1134a4887fe0909de2f21f6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f41525e3e1134a4887fe0909de2f21f6, State: Initialized, Role: FOLLOWER
I20260812 21:03:36.908941 15458 consensus_queue.cc:260] T 00000000000000000000000000000000 P f41525e3e1134a4887fe0909de2f21f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f41525e3e1134a4887fe0909de2f21f6" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 45761 } }
I20260812 21:03:36.909008 15458 raft_consensus.cc:388] T 00000000000000000000000000000000 P f41525e3e1134a4887fe0909de2f21f6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 21:03:36.909040 15458 raft_consensus.cc:482] T 00000000000000000000000000000000 P f41525e3e1134a4887fe0909de2f21f6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 21:03:36.909086 15458 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f41525e3e1134a4887fe0909de2f21f6 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:03:36.909678 15458 raft_consensus.cc:504] T 00000000000000000000000000000000 P f41525e3e1134a4887fe0909de2f21f6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f41525e3e1134a4887fe0909de2f21f6" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 45761 } }
I20260812 21:03:36.909778 15458 leader_election.cc:302] T 00000000000000000000000000000000 P f41525e3e1134a4887fe0909de2f21f6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f41525e3e1134a4887fe0909de2f21f6; no voters:
I20260812 21:03:36.909981 15458 leader_election.cc:288] T 00000000000000000000000000000000 P f41525e3e1134a4887fe0909de2f21f6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 21:03:36.910017 15463 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f41525e3e1134a4887fe0909de2f21f6 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:03:36.910192 15463 raft_consensus.cc:686] T 00000000000000000000000000000000 P f41525e3e1134a4887fe0909de2f21f6 [term 1 LEADER]: Becoming Leader. State: Replica: f41525e3e1134a4887fe0909de2f21f6, State: Running, Role: LEADER
I20260812 21:03:36.910250 15458 sys_catalog.cc:564] T 00000000000000000000000000000000 P f41525e3e1134a4887fe0909de2f21f6 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 21:03:36.910285 15463 consensus_queue.cc:237] T 00000000000000000000000000000000 P f41525e3e1134a4887fe0909de2f21f6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f41525e3e1134a4887fe0909de2f21f6" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 45761 } }
I20260812 21:03:36.910663 15464 sys_catalog.cc:455] T 00000000000000000000000000000000 P f41525e3e1134a4887fe0909de2f21f6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f41525e3e1134a4887fe0909de2f21f6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f41525e3e1134a4887fe0909de2f21f6" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 45761 } } }
I20260812 21:03:36.910667 15465 sys_catalog.cc:455] T 00000000000000000000000000000000 P f41525e3e1134a4887fe0909de2f21f6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f41525e3e1134a4887fe0909de2f21f6. Latest consensus state: current_term: 1 leader_uuid: "f41525e3e1134a4887fe0909de2f21f6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f41525e3e1134a4887fe0909de2f21f6" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 45761 } } }
I20260812 21:03:36.910763 15464 sys_catalog.cc:458] T 00000000000000000000000000000000 P f41525e3e1134a4887fe0909de2f21f6 [sys.catalog]: This master's current role is: LEADER
I20260812 21:03:36.910791 15465 sys_catalog.cc:458] T 00000000000000000000000000000000 P f41525e3e1134a4887fe0909de2f21f6 [sys.catalog]: This master's current role is: LEADER
I20260812 21:03:36.910979 15475 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 21:03:36.911198 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 15301
I20260812 21:03:36.911279 12282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568607354921-12282-0/minicluster-data/master-0/wal/instance
I20260812 21:03:36.911455 12282 external_mini_cluster.cc:1428] Setting key 6fbdc49a62a4472cc8398084cbb61a82
I20260812 21:03:36.911764 15475 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 21:03:36.913719 15475 catalog_manager.cc:1347] Generated new cluster ID: f157b367bd4747808e48fbfd0484ab89
I20260812 21:03:36.913765 15475 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 21:03:36 dist-test-slave-vpvm krb5kdc[15282](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568616, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.254.190@KRBTEST.COM
I20260812 21:03:36.923489 15491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:36.912174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35832 (local address 127.11.254.190:45761)
0812 21:03:36.912407 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0812 21:03:36.912417 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 21:03:36.912452 (+ 35us) server_negotiation.cc:407] Connection header received
0812 21:03:36.912572 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:03:36.912580 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:03:36.912650 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:03:36.912784 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:03:36.913691 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:36.915175 (+ 1484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:36.916235 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:36.916911 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:36.917026 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:36.919815 (+ 2789us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:03:36.919820 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:03:36.919821 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:03:36.919835 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:03:36.921689 (+ 1854us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:36.922114 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:36.922117 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:36.922118 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:36.922237 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:36.922425 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:36.922425 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:36.922426 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:36.922737 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:03:36.922906 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0812 21:03:36.923001 (+ 95us) server_negotiation.cc:299] Negotiation successful
0812 21:03:36.923274 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":182,"thread_start_us":85,"threads_started":1}
I20260812 21:03:36.923804 15475 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 21:03:36.924376 15475 catalog_manager.cc:1504] Loading token signing keys...
I20260812 21:03:36.931064 15475 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f41525e3e1134a4887fe0909de2f21f6: Generated new TSK 0
I20260812 21:03:36.931281 15475 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.254.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.254.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.254.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.129.keytab.
Entry for principal kudu/127.11.254.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.129.keytab.
I20260812 21:03:36.961027 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568607354921-12282-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568607354921-12282-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568607354921-12282-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568607354921-12282-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.254.129:0
--local_ip_for_outbound_sockets=127.11.254.129
--webserver_interface=127.11.254.129
--webserver_port=0
--tserver_master_addrs=127.11.254.190:45761
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.129.keytab
--principal=kudu/127.11.254.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.254.148:37257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568607354921-12282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568607354921-12282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568607354921-12282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:03:37.046638 15503 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:03:37.046842 15503 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:03:37.046868 15503 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:03:37.048028 15503 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:03:37.048084 15503 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:03:37.048110 15503 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260812 21:03:37.048122 15503 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 21:03:37.048153 15503 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.254.129
I20260812 21:03:37.049661 15503 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.254.148:37257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568607354921-12282-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568607354921-12282-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.129.keytab
--principal=kudu/127.11.254.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.254.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568607354921-12282-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.254.129
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.11.254.190:45761
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15503
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.254.129
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568607354921-12282-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:03:37.049892 15503 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:03:37.050158 15503 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:03:37.055187 15520 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:03:37.055181 15516 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:03:37.055284 15503 server_base.cc:1034] running on GCE node
W20260812 21:03:37.055181 15515 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:03:37.055640 15503 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:03:37.055912 15503 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:03:37.057054 15503 hybrid_clock.cc:648] HybridClock initialized: now 1786568617057033 us; error 32 us; skew 500 ppm
Aug 12 21:03:37 dist-test-slave-vpvm krb5kdc[15282](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568617, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:03:37.059653 15503 init.cc:377] Logged in from keytab as kudu/127.11.254.129@KRBTEST.COM (short username kudu)
I20260812 21:03:37.060101 15503 webserver.cc:460] Webserver started at http://127.11.254.129:41377/ using document root <none> and password file <none>
I20260812 21:03:37.060336 15503 fs_manager.cc:359] Metadata directory not provided
I20260812 21:03:37.060382 15503 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:03:37.060513 15503 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:03:37.061591 15503 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568607354921-12282-0/minicluster-data/ts-0/data/instance:
uuid: "92601044a2eb4651b9d3bb09cbe9d5b0"
format_stamp: "Formatted at 2026-08-12 21:03:37 on dist-test-slave-vpvm"
server_key: "ccbc36f763a81adebaf284e7238c3952"
server_key_iv: "f6dc1dbdbe4a2750f0366c7a767e674f"
server_key_version: "encryptionkey@0"
I20260812 21:03:37.061975 15503 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568607354921-12282-0/minicluster-data/ts-0/wal/instance:
uuid: "92601044a2eb4651b9d3bb09cbe9d5b0"
format_stamp: "Formatted at 2026-08-12 21:03:37 on dist-test-slave-vpvm"
server_key: "ccbc36f763a81adebaf284e7238c3952"
server_key_iv: "f6dc1dbdbe4a2750f0366c7a767e674f"
server_key_version: "encryptionkey@0"
I20260812 21:03:37.063388 15503 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 21:03:37.064158 15532 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:03:37.064340 15503 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 21:03:37.064422 15503 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568607354921-12282-0/minicluster-data/ts-0/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568607354921-12282-0/minicluster-data/ts-0/wal
uuid: "92601044a2eb4651b9d3bb09cbe9d5b0"
format_stamp: "Formatted at 2026-08-12 21:03:37 on dist-test-slave-vpvm"
server_key: "ccbc36f763a81adebaf284e7238c3952"
server_key_iv: "f6dc1dbdbe4a2750f0366c7a767e674f"
server_key_version: "encryptionkey@0"
I20260812 21:03:37.064481 15503 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568607354921-12282-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568607354921-12282-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568607354921-12282-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:03:37.083369 15503 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:03:37.085568 15503 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:03:37.085687 15503 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:03:37.086274 15503 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:03:37.086318 15503 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:03:37.086340 15503 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:03:37.086351 15503 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:03:37.093250 15503 rpc_server.cc:307] RPC server started. Bound to: 127.11.254.129:33109
Aug 12 21:03:37 dist-test-slave-vpvm krb5kdc[15282](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568617, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.129@KRBTEST.COM for kudu/127.11.254.190@KRBTEST.COM
I20260812 21:03:37.093411 15760 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.254.129:33109 every 8 connection(s)
I20260812 21:03:37.093683 15503 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568607354921-12282-0/minicluster-data/ts-0/data/info.pb
I20260812 21:03:37.096279 15491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:37.086739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.129:40379 (local address 127.11.254.190:45761)
0812 21:03:37.086819 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0812 21:03:37.086821 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:03:37.087054 (+ 233us) server_negotiation.cc:407] Connection header received
0812 21:03:37.087758 (+ 704us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:03:37.087759 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:03:37.087772 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:03:37.087831 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:03:37.088748 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:37.089560 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:37.091117 (+ 1557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:37.091436 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:37.091491 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:37.093925 (+ 2434us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:03:37.093927 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:03:37.093928 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:03:37.093940 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:03:37.095191 (+ 1251us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:37.095542 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:37.095543 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:37.095543 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:37.095594 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:37.095770 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:37.095770 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:37.095770 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:37.095870 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:03:37.095913 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0812 21:03:37.096133 (+ 220us) server_negotiation.cc:299] Negotiation successful
0812 21:03:37.096214 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 21:03:37.096431 15556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:37.086700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:45761 (local address 127.11.254.129:40379)
0812 21:03:37.086911 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0812 21:03:37.086960 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0812 21:03:37.087629 (+ 669us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:03:37.087939 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:03:37.087943 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:03:37.088326 (+ 383us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:03:37.088640 (+ 314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:37.088647 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:37.089660 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:37.089661 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:37.091010 (+ 1349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:37.091014 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:37.091516 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:37.091517 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:37.091669 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:37.092224 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:03:37.092233 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:03:37.093826 (+ 1593us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:03:37.095262 (+ 1436us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:03:37.095267 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:03:37.095269 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:03:37.095484 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:03:37.095639 (+ 155us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:03:37.095640 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:03:37.095640 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:03:37.095729 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:03:37.095909 (+ 180us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:03:37.095912 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:03:37.096022 (+ 110us) client_negotiation.cc:769] Sending connection context
0812 21:03:37.096115 (+ 93us) client_negotiation.cc:240] Negotiation successful
0812 21:03:37.096236 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":84,"threads_started":1}
I20260812 21:03:37.096925 15762 heartbeater.cc:347] Connected to a master server at 127.11.254.190:45761
I20260812 21:03:37.097016 15762 heartbeater.cc:464] Registering TS with master...
I20260812 21:03:37.097075 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 15503
I20260812 21:03:37.097172 12282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568607354921-12282-0/minicluster-data/ts-0/wal/instance
I20260812 21:03:37.097219 15762 heartbeater.cc:511] Master 127.11.254.190:45761 requested a full tablet report, sending...
I20260812 21:03:37.097354 12282 external_mini_cluster.cc:1428] Setting key e6961cdd498230f490d8aecd09a61378
I20260812 21:03:37.097754 15357 ts_manager.cc:194] Registered new tserver with Master: 92601044a2eb4651b9d3bb09cbe9d5b0 (127.11.254.129:33109)
I20260812 21:03:37.098532 15357 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.254.129@KRBTEST.COM'} at 127.11.254.129:40379
I20260812 21:03:37.103554 15491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:37.098246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.129:33389 (local address 127.11.254.190:45761)
0812 21:03:37.098281 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0812 21:03:37.098283 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:03:37.098398 (+ 115us) server_negotiation.cc:407] Connection header received
0812 21:03:37.098451 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:03:37.098452 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:03:37.098462 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:03:37.098506 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:03:37.098842 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:37.099502 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:37.100160 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:37.100364 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:37.100418 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:37.101242 (+ 824us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:03:37.101244 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:03:37.101244 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:03:37.101250 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:03:37.102547 (+ 1297us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:37.102848 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:37.102849 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:37.102850 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:37.102905 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:37.103063 (+ 158us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:37.103064 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:37.103065 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:37.103188 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:03:37.103244 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0812 21:03:37.103382 (+ 138us) server_negotiation.cc:299] Negotiation successful
0812 21:03:37.103462 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260812 21:03:37.103566 15775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:37.098202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:45761 (local address 127.11.254.129:33389)
0812 21:03:37.098337 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0812 21:03:37.098346 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 21:03:37.098399 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:03:37.098511 (+ 112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:03:37.098512 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:03:37.098708 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:03:37.098772 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:37.098774 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:37.099593 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:37.099594 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:37.100087 (+ 493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:37.100089 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:37.100423 (+ 334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:37.100424 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:37.100514 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:37.100897 (+ 383us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:03:37.100901 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:03:37.101165 (+ 264us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:03:37.102622 (+ 1457us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:03:37.102623 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:03:37.102623 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:03:37.102788 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:03:37.102955 (+ 167us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:03:37.102956 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:03:37.102956 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:03:37.103015 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:03:37.103246 (+ 231us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:03:37.103247 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:03:37.103294 (+ 47us) client_negotiation.cc:769] Sending connection context
0812 21:03:37.103353 (+ 59us) client_negotiation.cc:240] Negotiation successful
0812 21:03:37.103460 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":58,"threads_started":1}
WARNING: no policy specified for kudu/127.11.254.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.254.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.254.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.130.keytab.
Entry for principal kudu/127.11.254.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.130.keytab.
I20260812 21:03:37.125584 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568607354921-12282-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568607354921-12282-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568607354921-12282-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568607354921-12282-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.254.130:0
--local_ip_for_outbound_sockets=127.11.254.130
--webserver_interface=127.11.254.130
--webserver_port=0
--tserver_master_addrs=127.11.254.190:45761
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.130.keytab
--principal=kudu/127.11.254.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.254.148:37257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568607354921-12282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568607354921-12282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568607354921-12282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:03:37.209156 15778 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:03:37.209370 15778 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:03:37.209393 15778 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:03:37.210546 15778 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:03:37.210592 15778 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:03:37.210611 15778 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260812 21:03:37.210621 15778 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 21:03:37.210641 15778 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.254.130
I20260812 21:03:37.212186 15778 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.254.148:37257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568607354921-12282-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568607354921-12282-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.130.keytab
--principal=kudu/127.11.254.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.254.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568607354921-12282-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.254.130
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.11.254.190:45761
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15778
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.254.130
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568607354921-12282-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:03:37.212402 15778 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:03:37.212602 15778 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:03:37.217514 15786 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:03:37.217660 15778 server_base.cc:1034] running on GCE node
W20260812 21:03:37.217599 15784 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:03:37.217525 15788 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:03:37.217990 15778 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:03:37.218236 15778 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:03:37.219391 15778 hybrid_clock.cc:648] HybridClock initialized: now 1786568617219375 us; error 28 us; skew 500 ppm
Aug 12 21:03:37 dist-test-slave-vpvm krb5kdc[15282](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568617, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:03:37.221987 15778 init.cc:377] Logged in from keytab as kudu/127.11.254.130@KRBTEST.COM (short username kudu)
I20260812 21:03:37.222430 15778 webserver.cc:460] Webserver started at http://127.11.254.130:33617/ using document root <none> and password file <none>
I20260812 21:03:37.222643 15778 fs_manager.cc:359] Metadata directory not provided
I20260812 21:03:37.222684 15778 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:03:37.222777 15778 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:03:37.223910 15778 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568607354921-12282-0/minicluster-data/ts-1/data/instance:
uuid: "ca2f0c3dd5b0431a913c99ba5bfd2bd5"
format_stamp: "Formatted at 2026-08-12 21:03:37 on dist-test-slave-vpvm"
server_key: "c167e9c0cd7db456af11cba4785ea017"
server_key_iv: "fda2949b5f564e580f6c7026556719e5"
server_key_version: "encryptionkey@0"
I20260812 21:03:37.224228 15778 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568607354921-12282-0/minicluster-data/ts-1/wal/instance:
uuid: "ca2f0c3dd5b0431a913c99ba5bfd2bd5"
format_stamp: "Formatted at 2026-08-12 21:03:37 on dist-test-slave-vpvm"
server_key: "c167e9c0cd7db456af11cba4785ea017"
server_key_iv: "fda2949b5f564e580f6c7026556719e5"
server_key_version: "encryptionkey@0"
I20260812 21:03:37.225744 15778 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 21:03:37.226593 15805 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:03:37.226768 15778 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 21:03:37.226861 15778 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568607354921-12282-0/minicluster-data/ts-1/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568607354921-12282-0/minicluster-data/ts-1/wal
uuid: "ca2f0c3dd5b0431a913c99ba5bfd2bd5"
format_stamp: "Formatted at 2026-08-12 21:03:37 on dist-test-slave-vpvm"
server_key: "c167e9c0cd7db456af11cba4785ea017"
server_key_iv: "fda2949b5f564e580f6c7026556719e5"
server_key_version: "encryptionkey@0"
I20260812 21:03:37.226941 15778 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568607354921-12282-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568607354921-12282-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568607354921-12282-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:03:37.246397 15778 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:03:37.248765 15778 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:03:37.248947 15778 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:03:37.249485 15778 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:03:37.249527 15778 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:03:37.249553 15778 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:03:37.249564 15778 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 21:03:37 dist-test-slave-vpvm krb5kdc[15282](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568617, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.130@KRBTEST.COM for kudu/127.11.254.190@KRBTEST.COM
I20260812 21:03:37.256650 15778 rpc_server.cc:307] RPC server started. Bound to: 127.11.254.130:44949
I20260812 21:03:37.256726 16028 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.254.130:44949 every 8 connection(s)
I20260812 21:03:37.257122 15778 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568607354921-12282-0/minicluster-data/ts-1/data/info.pb
I20260812 21:03:37.259716 15491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:37.250071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.130:49241 (local address 127.11.254.190:45761)
0812 21:03:37.250124 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0812 21:03:37.250126 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:03:37.250330 (+ 204us) server_negotiation.cc:407] Connection header received
0812 21:03:37.250881 (+ 551us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:03:37.250882 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:03:37.250896 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:03:37.250948 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:03:37.251659 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:37.252451 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:37.253944 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:37.254281 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:37.254371 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:37.256837 (+ 2466us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:03:37.256838 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:03:37.256839 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:03:37.256849 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:03:37.258284 (+ 1435us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:37.258742 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:37.258743 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:37.258743 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:37.258814 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:37.259060 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:37.259062 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:37.259062 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:37.259199 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:03:37.259258 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0812 21:03:37.259583 (+ 325us) server_negotiation.cc:299] Negotiation successful
0812 21:03:37.259650 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260812 21:03:37.259894 15818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:37.250050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:45761 (local address 127.11.254.130:49241)
0812 21:03:37.250209 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0812 21:03:37.250246 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0812 21:03:37.250774 (+ 528us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:03:37.251017 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:03:37.251020 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:03:37.251289 (+ 269us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:03:37.251566 (+ 277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:37.251575 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:37.252543 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:37.252544 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:37.253823 (+ 1279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:37.253827 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:37.254403 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:37.254404 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:37.254515 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:37.254995 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:03:37.255005 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:03:37.256741 (+ 1736us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:03:37.258361 (+ 1620us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:03:37.258368 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:03:37.258369 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:03:37.258638 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:03:37.258872 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:03:37.258873 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:03:37.258873 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:03:37.258972 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:03:37.259291 (+ 319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:03:37.259299 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:03:37.259470 (+ 171us) client_negotiation.cc:769] Sending connection context
0812 21:03:37.259564 (+ 94us) client_negotiation.cc:240] Negotiation successful
0812 21:03:37.259697 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":80,"threads_started":1}
I20260812 21:03:37.260389 16029 heartbeater.cc:347] Connected to a master server at 127.11.254.190:45761
I20260812 21:03:37.260470 16029 heartbeater.cc:464] Registering TS with master...
I20260812 21:03:37.260653 16029 heartbeater.cc:511] Master 127.11.254.190:45761 requested a full tablet report, sending...
I20260812 21:03:37.261056 15361 ts_manager.cc:194] Registered new tserver with Master: ca2f0c3dd5b0431a913c99ba5bfd2bd5 (127.11.254.130:44949)
I20260812 21:03:37.261570 15361 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.254.130@KRBTEST.COM'} at 127.11.254.130:49241
I20260812 21:03:37.261577 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 15778
I20260812 21:03:37.261672 12282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568607354921-12282-0/minicluster-data/ts-1/wal/instance
I20260812 21:03:37.261859 12282 external_mini_cluster.cc:1428] Setting key eb4dc3eae7579e7c853be18e52748a3d
I20260812 21:03:37.269367 16042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:37.261568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:45761 (local address 127.11.254.130:33383)
0812 21:03:37.262147 (+ 579us) negotiation.cc:107] Waiting for socket to connect
0812 21:03:37.262158 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 21:03:37.262231 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:03:37.262388 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:03:37.262389 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:03:37.264173 (+ 1784us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:03:37.264228 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:37.264231 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:37.264876 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:37.264876 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:37.265380 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:37.265382 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:37.265780 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:37.265781 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:37.265849 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:37.266318 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:03:37.266323 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:03:37.266633 (+ 310us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:03:37.268384 (+ 1751us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:03:37.268385 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:03:37.268385 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:03:37.268558 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:03:37.268753 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:03:37.268753 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:03:37.268754 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:03:37.268809 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:03:37.269132 (+ 323us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:03:37.269133 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:03:37.269172 (+ 39us) client_negotiation.cc:769] Sending connection context
0812 21:03:37.269218 (+ 46us) client_negotiation.cc:240] Negotiation successful
0812 21:03:37.269296 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":556,"thread_start_us":81,"threads_started":1}
I20260812 21:03:37.269414 15491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:37.262049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.130:33383 (local address 127.11.254.190:45761)
0812 21:03:37.262090 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0812 21:03:37.262092 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:03:37.262206 (+ 114us) server_negotiation.cc:407] Connection header received
0812 21:03:37.262296 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:03:37.262297 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:03:37.262306 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:03:37.262333 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:03:37.264316 (+ 1983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:37.264789 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:37.265451 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:37.265698 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:37.265761 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:37.266726 (+ 965us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:03:37.266727 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:03:37.266728 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:03:37.266735 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:03:37.268310 (+ 1575us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:37.268634 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:37.268635 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:37.268636 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:37.268695 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:37.268860 (+ 165us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:37.268861 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:37.268861 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:37.269072 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:03:37.269143 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0812 21:03:37.269246 (+ 103us) server_negotiation.cc:299] Negotiation successful
0812 21:03:37.269325 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
WARNING: no policy specified for kudu/127.11.254.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.254.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.254.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.131.keytab.
Entry for principal kudu/127.11.254.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.131.keytab.
I20260812 21:03:37.288901 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568607354921-12282-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568607354921-12282-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568607354921-12282-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568607354921-12282-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.254.131:0
--local_ip_for_outbound_sockets=127.11.254.131
--webserver_interface=127.11.254.131
--webserver_port=0
--tserver_master_addrs=127.11.254.190:45761
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.131.keytab
--principal=kudu/127.11.254.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.254.148:37257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568607354921-12282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568607354921-12282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568607354921-12282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:03:37.374897 16045 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:03:37.375125 16045 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:03:37.375164 16045 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:03:37.376338 16045 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:03:37.376382 16045 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:03:37.376411 16045 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260812 21:03:37.376426 16045 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 21:03:37.376451 16045 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.254.131
I20260812 21:03:37.378002 16045 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.254.148:37257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568607354921-12282-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568607354921-12282-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.131.keytab
--principal=kudu/127.11.254.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.254.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568607354921-12282-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.254.131
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.11.254.190:45761
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16045
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.254.131
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568607354921-12282-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:03:37.378257 16045 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:03:37.378475 16045 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:03:37.383456 16055 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:03:37.383471 16060 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:03:37.383493 16054 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:03:37.383671 16045 server_base.cc:1034] running on GCE node
I20260812 21:03:37.383843 16045 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:03:37.384079 16045 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:03:37.385231 16045 hybrid_clock.cc:648] HybridClock initialized: now 1786568617385213 us; error 28 us; skew 500 ppm
Aug 12 21:03:37 dist-test-slave-vpvm krb5kdc[15282](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568617, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:03:37.387866 16045 init.cc:377] Logged in from keytab as kudu/127.11.254.131@KRBTEST.COM (short username kudu)
I20260812 21:03:37.388336 16045 webserver.cc:460] Webserver started at http://127.11.254.131:37335/ using document root <none> and password file <none>
I20260812 21:03:37.388556 16045 fs_manager.cc:359] Metadata directory not provided
I20260812 21:03:37.388612 16045 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:03:37.388744 16045 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:03:37.389823 16045 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568607354921-12282-0/minicluster-data/ts-2/data/instance:
uuid: "cb92e98ad74b42af916785483c36e382"
format_stamp: "Formatted at 2026-08-12 21:03:37 on dist-test-slave-vpvm"
server_key: "def1a8e29a89082a5b49d80844b5f081"
server_key_iv: "0de61b1098ef475e309d1a7178567b41"
server_key_version: "encryptionkey@0"
I20260812 21:03:37.390228 16045 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568607354921-12282-0/minicluster-data/ts-2/wal/instance:
uuid: "cb92e98ad74b42af916785483c36e382"
format_stamp: "Formatted at 2026-08-12 21:03:37 on dist-test-slave-vpvm"
server_key: "def1a8e29a89082a5b49d80844b5f081"
server_key_iv: "0de61b1098ef475e309d1a7178567b41"
server_key_version: "encryptionkey@0"
I20260812 21:03:37.391510 16045 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 21:03:37.392264 16068 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:03:37.392412 16045 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 21:03:37.392498 16045 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568607354921-12282-0/minicluster-data/ts-2/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568607354921-12282-0/minicluster-data/ts-2/wal
uuid: "cb92e98ad74b42af916785483c36e382"
format_stamp: "Formatted at 2026-08-12 21:03:37 on dist-test-slave-vpvm"
server_key: "def1a8e29a89082a5b49d80844b5f081"
server_key_iv: "0de61b1098ef475e309d1a7178567b41"
server_key_version: "encryptionkey@0"
I20260812 21:03:37.392596 16045 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568607354921-12282-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568607354921-12282-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568607354921-12282-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:03:37.410323 16045 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:03:37.412715 16045 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:03:37.412860 16045 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:03:37.413414 16045 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:03:37.413460 16045 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:03:37.413491 16045 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:03:37.413507 16045 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:03:37.419909 16045 rpc_server.cc:307] RPC server started. Bound to: 127.11.254.131:35785
I20260812 21:03:37.420012 16288 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.254.131:35785 every 8 connection(s)
Aug 12 21:03:37 dist-test-slave-vpvm krb5kdc[15282](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568617, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.131@KRBTEST.COM for kudu/127.11.254.190@KRBTEST.COM
I20260812 21:03:37.420293 16045 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568607354921-12282-0/minicluster-data/ts-2/data/info.pb
I20260812 21:03:37.423310 15491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:37.413822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.131:43753 (local address 127.11.254.190:45761)
0812 21:03:37.413884 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0812 21:03:37.413887 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 21:03:37.414122 (+ 235us) server_negotiation.cc:407] Connection header received
0812 21:03:37.414644 (+ 522us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:03:37.414645 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:03:37.414659 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:03:37.414711 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:03:37.415372 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:37.416162 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:37.417777 (+ 1615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:37.418113 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:37.418176 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:37.420784 (+ 2608us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:03:37.420786 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:03:37.420786 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:03:37.420793 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:03:37.422078 (+ 1285us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:37.422461 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:37.422462 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:37.422463 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:37.422529 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:37.422724 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:37.422725 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:37.422725 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:37.422893 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:03:37.422952 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0812 21:03:37.423157 (+ 205us) server_negotiation.cc:299] Negotiation successful
0812 21:03:37.423243 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260812 21:03:37.423511 16089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:37.413800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:45761 (local address 127.11.254.131:43753)
0812 21:03:37.413993 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0812 21:03:37.414034 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 21:03:37.414540 (+ 506us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:03:37.414782 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:03:37.414786 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:03:37.415026 (+ 240us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:03:37.415276 (+ 250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:37.415283 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:37.416249 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:37.416251 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:37.417658 (+ 1407us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:37.417662 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:37.418252 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:37.418254 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:37.418397 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:37.418950 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:03:37.418961 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:03:37.420703 (+ 1742us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:03:37.422159 (+ 1456us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:03:37.422165 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:03:37.422166 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:03:37.422373 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:03:37.422589 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:03:37.422590 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:03:37.422590 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:03:37.422688 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:03:37.422933 (+ 245us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:03:37.422936 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:03:37.423045 (+ 109us) client_negotiation.cc:769] Sending connection context
0812 21:03:37.423123 (+ 78us) client_negotiation.cc:240] Negotiation successful
0812 21:03:37.423281 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":88,"threads_started":1}
I20260812 21:03:37.424031 16290 heartbeater.cc:347] Connected to a master server at 127.11.254.190:45761
I20260812 21:03:37.424114 16290 heartbeater.cc:464] Registering TS with master...
I20260812 21:03:37.424280 16290 heartbeater.cc:511] Master 127.11.254.190:45761 requested a full tablet report, sending...
I20260812 21:03:37.424652 15357 ts_manager.cc:194] Registered new tserver with Master: cb92e98ad74b42af916785483c36e382 (127.11.254.131:35785)
I20260812 21:03:37.425118 15357 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.254.131@KRBTEST.COM'} at 127.11.254.131:43753
I20260812 21:03:37.426968 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 16045
I20260812 21:03:37.427060 12282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568607354921-12282-0/minicluster-data/ts-2/wal/instance
I20260812 21:03:37.427263 12282 external_mini_cluster.cc:1428] Setting key f4db82c8b0a322007163f2226e9fdaab
I20260812 21:03:37.428110 12282 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 21:03:37.430507 15491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:37.425092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.131:38279 (local address 127.11.254.190:45761)
0812 21:03:37.425137 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0812 21:03:37.425139 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:03:37.425253 (+ 114us) server_negotiation.cc:407] Connection header received
0812 21:03:37.425276 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:03:37.425277 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:03:37.425287 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:03:37.425334 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:03:37.425673 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:37.426263 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:37.426925 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:37.427210 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:37.427278 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:37.428200 (+ 922us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:03:37.428201 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:03:37.428202 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:03:37.428208 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:03:37.429439 (+ 1231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:37.429784 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:37.429785 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:37.429785 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:37.429841 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:37.430029 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:37.430029 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:37.430030 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:37.430143 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:03:37.430203 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0812 21:03:37.430334 (+ 131us) server_negotiation.cc:299] Negotiation successful
0812 21:03:37.430406 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 21:03:37.430539 16303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:37.424972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:45761 (local address 127.11.254.131:38279)
0812 21:03:37.425139 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0812 21:03:37.425151 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 21:03:37.425215 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:03:37.425353 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:03:37.425354 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:03:37.425537 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:03:37.425597 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:37.425598 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:37.426342 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:37.426343 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:37.426856 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:37.426857 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:37.427279 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:37.427280 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:37.427345 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:37.427781 (+ 436us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:03:37.427786 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:03:37.428130 (+ 344us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:03:37.429508 (+ 1378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:03:37.429509 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:03:37.429509 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:03:37.429724 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:03:37.429894 (+ 170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:03:37.429895 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:03:37.429895 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:03:37.429983 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:03:37.430198 (+ 215us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:03:37.430199 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:03:37.430244 (+ 45us) client_negotiation.cc:769] Sending connection context
0812 21:03:37.430305 (+ 61us) client_negotiation.cc:240] Negotiation successful
0812 21:03:37.430409 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":73,"threads_started":1}
Aug 12 21:03:37 dist-test-slave-vpvm krb5kdc[15282](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568617, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 21:03:37 dist-test-slave-vpvm krb5kdc[15282](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568617, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.254.190@KRBTEST.COM
I20260812 21:03:37.451049 15491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:37.444815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35844 (local address 127.11.254.190:45761)
0812 21:03:37.444872 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0812 21:03:37.444874 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:03:37.444883 (+ 9us) server_negotiation.cc:407] Connection header received
0812 21:03:37.444908 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:03:37.444909 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:03:37.444918 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:03:37.444972 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:03:37.445469 (+ 497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:37.445988 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:37.446680 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:37.446860 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:37.446904 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:37.448945 (+ 2041us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:03:37.448946 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:03:37.448946 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:03:37.448954 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:03:37.450223 (+ 1269us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:37.450489 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:37.450489 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:37.450490 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:37.450561 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:37.450686 (+ 125us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:37.450686 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:37.450687 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:37.450786 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:03:37.450851 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0812 21:03:37.450924 (+ 73us) server_negotiation.cc:299] Negotiation successful
0812 21:03:37.450979 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260812 21:03:37.452611 15357 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35844:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 21:03:37.454239 15357 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 21:03:37.461472 16332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:37.456496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.129:33109 (local address 127.0.0.1:40914)
0812 21:03:37.457006 (+ 510us) negotiation.cc:107] Waiting for socket to connect
0812 21:03:37.457016 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 21:03:37.457090 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:03:37.457386 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:03:37.457387 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:03:37.457389 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:03:37.457422 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:37.457422 (+ 0us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:37.458775 (+ 1353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:37.458776 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:37.460160 (+ 1384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:37.460163 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:37.461133 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:37.461134 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:37.461208 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:37.461219 (+ 11us) client_negotiation.cc:769] Sending connection context
0812 21:03:37.461281 (+ 62us) client_negotiation.cc:240] Negotiation successful
0812 21:03:37.461388 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":489,"mutex_wait_us":8,"thread_start_us":87,"threads_started":1}
I20260812 21:03:37.461491 16330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:37.456626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40914 (local address 127.11.254.129:33109)
0812 21:03:37.456826 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0812 21:03:37.456831 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 21:03:37.457093 (+ 262us) server_negotiation.cc:407] Connection header received
0812 21:03:37.457222 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:03:37.457224 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:03:37.457242 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:03:37.457316 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:03:37.457484 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:37.458673 (+ 1189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:37.460250 (+ 1577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:37.461049 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:37.461137 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:37.461197 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0812 21:03:37.461311 (+ 114us) server_negotiation.cc:299] Negotiation successful
0812 21:03:37.461398 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":96,"threads_started":1}
I20260812 21:03:37.462095 16326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:37.456720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.130:44949 (local address 127.0.0.1:53566)
0812 21:03:37.457122 (+ 402us) negotiation.cc:107] Waiting for socket to connect
0812 21:03:37.457130 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0812 21:03:37.457174 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:03:37.457376 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:03:37.457377 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:03:37.457379 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:03:37.457421 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:37.457422 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:37.459689 (+ 2267us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:37.459690 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:37.460514 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:37.460517 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:37.461816 (+ 1299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:37.461817 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:37.461893 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:37.461897 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 21:03:37.461981 (+ 84us) client_negotiation.cc:240] Negotiation successful
0812 21:03:37.462023 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":382,"thread_start_us":71,"threads_started":1}
I20260812 21:03:37.462311 16335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:37.456842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53566 (local address 127.11.254.130:44949)
0812 21:03:37.457000 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0812 21:03:37.457004 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 21:03:37.457231 (+ 227us) server_negotiation.cc:407] Connection header received
0812 21:03:37.457282 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:03:37.457284 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:03:37.457300 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:03:37.457367 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:03:37.458775 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:37.459617 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:37.460643 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:37.461339 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:37.461719 (+ 380us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:37.461782 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0812 21:03:37.462001 (+ 219us) server_negotiation.cc:299] Negotiation successful
0812 21:03:37.462053 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":78,"threads_started":1}
I20260812 21:03:37.462416 16324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:37.456496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.131:35785 (local address 127.0.0.1:57404)
0812 21:03:37.456886 (+ 390us) negotiation.cc:107] Waiting for socket to connect
0812 21:03:37.456922 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0812 21:03:37.456994 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:03:37.457180 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:03:37.457183 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:03:37.457185 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:03:37.457260 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:37.457263 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:37.458774 (+ 1511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:37.458775 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:37.460161 (+ 1386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:37.460163 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:37.462210 (+ 2047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:37.462211 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:37.462259 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:37.462261 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 21:03:37.462307 (+ 46us) client_negotiation.cc:240] Negotiation successful
0812 21:03:37.462349 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":364,"mutex_wait_us":95,"thread_start_us":85,"threads_started":1}
I20260812 21:03:37.462471 16327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:37.456632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57404 (local address 127.11.254.131:35785)
0812 21:03:37.456826 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0812 21:03:37.456831 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 21:03:37.456990 (+ 159us) server_negotiation.cc:407] Connection header received
0812 21:03:37.457078 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:03:37.457080 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:03:37.457097 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:03:37.457176 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:03:37.457517 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:37.458673 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:37.461238 (+ 2565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:37.462050 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:37.462123 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:37.462175 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0812 21:03:37.462337 (+ 162us) server_negotiation.cc:299] Negotiation successful
0812 21:03:37.462381 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":91,"threads_started":1}
I20260812 21:03:37.462579 15630 tablet_service.cc:1467] Processing CreateTablet for tablet 0c21bbe9e74d42c88248e815288e5909 (DEFAULT_TABLE table=test-table [id=bb10a9699d364b92ac606dccb3272709]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:03:37.462949 15630 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0c21bbe9e74d42c88248e815288e5909. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:03:37.465895 16338 tablet_bootstrap.cc:492] T 0c21bbe9e74d42c88248e815288e5909 P 92601044a2eb4651b9d3bb09cbe9d5b0: Bootstrap starting.
I20260812 21:03:37.466547 16338 tablet_bootstrap.cc:654] T 0c21bbe9e74d42c88248e815288e5909 P 92601044a2eb4651b9d3bb09cbe9d5b0: Neither blocks nor log segments found. Creating new log.
I20260812 21:03:37.466571 16169 tablet_service.cc:1467] Processing CreateTablet for tablet 0c21bbe9e74d42c88248e815288e5909 (DEFAULT_TABLE table=test-table [id=bb10a9699d364b92ac606dccb3272709]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:03:37.466655 15906 tablet_service.cc:1467] Processing CreateTablet for tablet 0c21bbe9e74d42c88248e815288e5909 (DEFAULT_TABLE table=test-table [id=bb10a9699d364b92ac606dccb3272709]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:03:37.466886 16338 log.cc:824] T 0c21bbe9e74d42c88248e815288e5909 P 92601044a2eb4651b9d3bb09cbe9d5b0: Log is configured to *not* fsync() on all Append() calls
I20260812 21:03:37.466949 16169 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0c21bbe9e74d42c88248e815288e5909. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:03:37.466949 15906 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0c21bbe9e74d42c88248e815288e5909. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:03:37.467757 16338 tablet_bootstrap.cc:492] T 0c21bbe9e74d42c88248e815288e5909 P 92601044a2eb4651b9d3bb09cbe9d5b0: No bootstrap required, opened a new log
I20260812 21:03:37.467824 16338 ts_tablet_manager.cc:1397] T 0c21bbe9e74d42c88248e815288e5909 P 92601044a2eb4651b9d3bb09cbe9d5b0: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 21:03:37.470073 16338 raft_consensus.cc:348] T 0c21bbe9e74d42c88248e815288e5909 P 92601044a2eb4651b9d3bb09cbe9d5b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92601044a2eb4651b9d3bb09cbe9d5b0" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 33109 } } peers { permanent_uuid: "cb92e98ad74b42af916785483c36e382" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 35785 } } peers { permanent_uuid: "ca2f0c3dd5b0431a913c99ba5bfd2bd5" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 44949 } }
I20260812 21:03:37.470232 16338 raft_consensus.cc:374] T 0c21bbe9e74d42c88248e815288e5909 P 92601044a2eb4651b9d3bb09cbe9d5b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:03:37.470266 16338 raft_consensus.cc:729] T 0c21bbe9e74d42c88248e815288e5909 P 92601044a2eb4651b9d3bb09cbe9d5b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92601044a2eb4651b9d3bb09cbe9d5b0, State: Initialized, Role: FOLLOWER
I20260812 21:03:37.470410 16338 consensus_queue.cc:260] T 0c21bbe9e74d42c88248e815288e5909 P 92601044a2eb4651b9d3bb09cbe9d5b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92601044a2eb4651b9d3bb09cbe9d5b0" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 33109 } } peers { permanent_uuid: "cb92e98ad74b42af916785483c36e382" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 35785 } } peers { permanent_uuid: "ca2f0c3dd5b0431a913c99ba5bfd2bd5" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 44949 } }
I20260812 21:03:37.470682 16338 ts_tablet_manager.cc:1428] T 0c21bbe9e74d42c88248e815288e5909 P 92601044a2eb4651b9d3bb09cbe9d5b0: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 21:03:37.470809 16342 tablet_bootstrap.cc:492] T 0c21bbe9e74d42c88248e815288e5909 P cb92e98ad74b42af916785483c36e382: Bootstrap starting.
I20260812 21:03:37.470836 15762 heartbeater.cc:503] Master 127.11.254.190:45761 was elected leader, sending a full tablet report...
I20260812 21:03:37.471679 16342 tablet_bootstrap.cc:654] T 0c21bbe9e74d42c88248e815288e5909 P cb92e98ad74b42af916785483c36e382: Neither blocks nor log segments found. Creating new log.
I20260812 21:03:37.471925 16340 tablet_bootstrap.cc:492] T 0c21bbe9e74d42c88248e815288e5909 P ca2f0c3dd5b0431a913c99ba5bfd2bd5: Bootstrap starting.
I20260812 21:03:37.472437 16340 tablet_bootstrap.cc:654] T 0c21bbe9e74d42c88248e815288e5909 P ca2f0c3dd5b0431a913c99ba5bfd2bd5: Neither blocks nor log segments found. Creating new log.
I20260812 21:03:37.472661 16340 log.cc:824] T 0c21bbe9e74d42c88248e815288e5909 P ca2f0c3dd5b0431a913c99ba5bfd2bd5: Log is configured to *not* fsync() on all Append() calls
I20260812 21:03:37.473079 16342 log.cc:824] T 0c21bbe9e74d42c88248e815288e5909 P cb92e98ad74b42af916785483c36e382: Log is configured to *not* fsync() on all Append() calls
I20260812 21:03:37.473310 16340 tablet_bootstrap.cc:492] T 0c21bbe9e74d42c88248e815288e5909 P ca2f0c3dd5b0431a913c99ba5bfd2bd5: No bootstrap required, opened a new log
I20260812 21:03:37.473369 16340 ts_tablet_manager.cc:1397] T 0c21bbe9e74d42c88248e815288e5909 P ca2f0c3dd5b0431a913c99ba5bfd2bd5: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 21:03:37.473701 16342 tablet_bootstrap.cc:492] T 0c21bbe9e74d42c88248e815288e5909 P cb92e98ad74b42af916785483c36e382: No bootstrap required, opened a new log
I20260812 21:03:37.473753 16342 ts_tablet_manager.cc:1397] T 0c21bbe9e74d42c88248e815288e5909 P cb92e98ad74b42af916785483c36e382: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260812 21:03:37.474946 16340 raft_consensus.cc:348] T 0c21bbe9e74d42c88248e815288e5909 P ca2f0c3dd5b0431a913c99ba5bfd2bd5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92601044a2eb4651b9d3bb09cbe9d5b0" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 33109 } } peers { permanent_uuid: "cb92e98ad74b42af916785483c36e382" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 35785 } } peers { permanent_uuid: "ca2f0c3dd5b0431a913c99ba5bfd2bd5" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 44949 } }
I20260812 21:03:37.475077 16340 raft_consensus.cc:374] T 0c21bbe9e74d42c88248e815288e5909 P ca2f0c3dd5b0431a913c99ba5bfd2bd5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:03:37.475111 16340 raft_consensus.cc:729] T 0c21bbe9e74d42c88248e815288e5909 P ca2f0c3dd5b0431a913c99ba5bfd2bd5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca2f0c3dd5b0431a913c99ba5bfd2bd5, State: Initialized, Role: FOLLOWER
I20260812 21:03:37.475221 16340 consensus_queue.cc:260] T 0c21bbe9e74d42c88248e815288e5909 P ca2f0c3dd5b0431a913c99ba5bfd2bd5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92601044a2eb4651b9d3bb09cbe9d5b0" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 33109 } } peers { permanent_uuid: "cb92e98ad74b42af916785483c36e382" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 35785 } } peers { permanent_uuid: "ca2f0c3dd5b0431a913c99ba5bfd2bd5" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 44949 } }
I20260812 21:03:37.475286 16342 raft_consensus.cc:348] T 0c21bbe9e74d42c88248e815288e5909 P cb92e98ad74b42af916785483c36e382 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92601044a2eb4651b9d3bb09cbe9d5b0" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 33109 } } peers { permanent_uuid: "cb92e98ad74b42af916785483c36e382" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 35785 } } peers { permanent_uuid: "ca2f0c3dd5b0431a913c99ba5bfd2bd5" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 44949 } }
I20260812 21:03:37.475437 16340 ts_tablet_manager.cc:1428] T 0c21bbe9e74d42c88248e815288e5909 P ca2f0c3dd5b0431a913c99ba5bfd2bd5: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260812 21:03:37.475442 16342 raft_consensus.cc:374] T 0c21bbe9e74d42c88248e815288e5909 P cb92e98ad74b42af916785483c36e382 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:03:37.475481 16342 raft_consensus.cc:729] T 0c21bbe9e74d42c88248e815288e5909 P cb92e98ad74b42af916785483c36e382 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb92e98ad74b42af916785483c36e382, State: Initialized, Role: FOLLOWER
I20260812 21:03:37.475492 16029 heartbeater.cc:503] Master 127.11.254.190:45761 was elected leader, sending a full tablet report...
I20260812 21:03:37.475595 16342 consensus_queue.cc:260] T 0c21bbe9e74d42c88248e815288e5909 P cb92e98ad74b42af916785483c36e382 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92601044a2eb4651b9d3bb09cbe9d5b0" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 33109 } } peers { permanent_uuid: "cb92e98ad74b42af916785483c36e382" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 35785 } } peers { permanent_uuid: "ca2f0c3dd5b0431a913c99ba5bfd2bd5" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 44949 } }
I20260812 21:03:37.475845 16342 ts_tablet_manager.cc:1428] T 0c21bbe9e74d42c88248e815288e5909 P cb92e98ad74b42af916785483c36e382: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260812 21:03:37.475875 16290 heartbeater.cc:503] Master 127.11.254.190:45761 was elected leader, sending a full tablet report...
W20260812 21:03:37.507704 16030 tablet.cc:2369] T 0c21bbe9e74d42c88248e815288e5909 P ca2f0c3dd5b0431a913c99ba5bfd2bd5: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 21:03:37.594533 15764 tablet.cc:2369] T 0c21bbe9e74d42c88248e815288e5909 P 92601044a2eb4651b9d3bb09cbe9d5b0: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 21:03:37.670852 16291 tablet.cc:2369] T 0c21bbe9e74d42c88248e815288e5909 P cb92e98ad74b42af916785483c36e382: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 21:03:37.872805 16350 raft_consensus.cc:482] T 0c21bbe9e74d42c88248e815288e5909 P cb92e98ad74b42af916785483c36e382 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 21:03:37.872937 16350 raft_consensus.cc:504] T 0c21bbe9e74d42c88248e815288e5909 P cb92e98ad74b42af916785483c36e382 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92601044a2eb4651b9d3bb09cbe9d5b0" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 33109 } } peers { permanent_uuid: "cb92e98ad74b42af916785483c36e382" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 35785 } } peers { permanent_uuid: "ca2f0c3dd5b0431a913c99ba5bfd2bd5" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 44949 } }
I20260812 21:03:37.873356 16350 leader_election.cc:288] T 0c21bbe9e74d42c88248e815288e5909 P cb92e98ad74b42af916785483c36e382 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 92601044a2eb4651b9d3bb09cbe9d5b0 (127.11.254.129:33109), ca2f0c3dd5b0431a913c99ba5bfd2bd5 (127.11.254.130:44949)
I20260812 21:03:37.876103 16330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:37.873651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.131:40987 (local address 127.11.254.129:33109)
0812 21:03:37.873711 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0812 21:03:37.873713 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:03:37.873723 (+ 10us) server_negotiation.cc:407] Connection header received
0812 21:03:37.873754 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:03:37.873756 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:03:37.873765 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:03:37.873812 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:03:37.873975 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:37.874478 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:37.875427 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:37.875784 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:37.875846 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:37.875855 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 21:03:37.875998 (+ 143us) server_negotiation.cc:299] Negotiation successful
0812 21:03:37.876045 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260812 21:03:37.876101 16089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:37.873592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.129:33109 (local address 127.11.254.131:40987)
0812 21:03:37.873641 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0812 21:03:37.873652 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 21:03:37.873699 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:03:37.873826 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:03:37.873826 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:03:37.873828 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:03:37.873896 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:37.873898 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:37.874604 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:37.874605 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:37.875335 (+ 730us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:37.875336 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:37.875855 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:37.875857 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:37.875934 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:37.875938 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 21:03:37.875983 (+ 45us) client_negotiation.cc:240] Negotiation successful
0812 21:03:37.876022 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260812 21:03:37.876390 16335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:37.873680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.131:53957 (local address 127.11.254.130:44949)
0812 21:03:37.873731 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0812 21:03:37.873733 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:03:37.873967 (+ 234us) server_negotiation.cc:407] Connection header received
0812 21:03:37.874059 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:03:37.874060 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:03:37.874069 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:03:37.874118 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:03:37.874271 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:37.874771 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:37.875667 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:37.876026 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:37.876087 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:37.876095 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 21:03:37.876285 (+ 190us) server_negotiation.cc:299] Negotiation successful
0812 21:03:37.876326 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260812 21:03:37.876438 15670 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c21bbe9e74d42c88248e815288e5909" candidate_uuid: "cb92e98ad74b42af916785483c36e382" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92601044a2eb4651b9d3bb09cbe9d5b0" is_pre_election: true
I20260812 21:03:37.876624 15670 raft_consensus.cc:2454] T 0c21bbe9e74d42c88248e815288e5909 P 92601044a2eb4651b9d3bb09cbe9d5b0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb92e98ad74b42af916785483c36e382 in term 0.
I20260812 21:03:37.876652 16352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:37.873593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.130:44949 (local address 127.11.254.131:53957)
0812 21:03:37.873867 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0812 21:03:37.873876 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 21:03:37.873992 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:03:37.874119 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:03:37.874120 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:03:37.874121 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:03:37.874211 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:37.874213 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:37.874854 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:37.874856 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:37.875578 (+ 722us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:37.875580 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:37.876118 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:37.876119 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:37.876192 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:37.876197 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 21:03:37.876511 (+ 314us) client_negotiation.cc:240] Negotiation successful
0812 21:03:37.876555 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"mutex_wait_us":92,"thread_start_us":114,"threads_started":1}
I20260812 21:03:37.876878 16069 leader_election.cc:302] T 0c21bbe9e74d42c88248e815288e5909 P cb92e98ad74b42af916785483c36e382 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92601044a2eb4651b9d3bb09cbe9d5b0, cb92e98ad74b42af916785483c36e382; no voters:
I20260812 21:03:37.877017 16350 raft_consensus.cc:2781] T 0c21bbe9e74d42c88248e815288e5909 P cb92e98ad74b42af916785483c36e382 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 21:03:37.876999 15944 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c21bbe9e74d42c88248e815288e5909" candidate_uuid: "cb92e98ad74b42af916785483c36e382" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca2f0c3dd5b0431a913c99ba5bfd2bd5" is_pre_election: true
I20260812 21:03:37.877089 16350 raft_consensus.cc:482] T 0c21bbe9e74d42c88248e815288e5909 P cb92e98ad74b42af916785483c36e382 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 21:03:37.877111 16350 raft_consensus.cc:3037] T 0c21bbe9e74d42c88248e815288e5909 P cb92e98ad74b42af916785483c36e382 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:03:37.877144 15944 raft_consensus.cc:2454] T 0c21bbe9e74d42c88248e815288e5909 P ca2f0c3dd5b0431a913c99ba5bfd2bd5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb92e98ad74b42af916785483c36e382 in term 0.
I20260812 21:03:37.878343 16350 raft_consensus.cc:504] T 0c21bbe9e74d42c88248e815288e5909 P cb92e98ad74b42af916785483c36e382 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92601044a2eb4651b9d3bb09cbe9d5b0" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 33109 } } peers { permanent_uuid: "cb92e98ad74b42af916785483c36e382" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 35785 } } peers { permanent_uuid: "ca2f0c3dd5b0431a913c99ba5bfd2bd5" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 44949 } }
I20260812 21:03:37.878469 16350 leader_election.cc:288] T 0c21bbe9e74d42c88248e815288e5909 P cb92e98ad74b42af916785483c36e382 [CANDIDATE]: Term 1 election: Requested vote from peers 92601044a2eb4651b9d3bb09cbe9d5b0 (127.11.254.129:33109), ca2f0c3dd5b0431a913c99ba5bfd2bd5 (127.11.254.130:44949)
I20260812 21:03:37.878640 15670 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c21bbe9e74d42c88248e815288e5909" candidate_uuid: "cb92e98ad74b42af916785483c36e382" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92601044a2eb4651b9d3bb09cbe9d5b0"
I20260812 21:03:37.878674 15944 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c21bbe9e74d42c88248e815288e5909" candidate_uuid: "cb92e98ad74b42af916785483c36e382" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca2f0c3dd5b0431a913c99ba5bfd2bd5"
I20260812 21:03:37.878728 15670 raft_consensus.cc:3037] T 0c21bbe9e74d42c88248e815288e5909 P 92601044a2eb4651b9d3bb09cbe9d5b0 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:03:37.878736 15944 raft_consensus.cc:3037] T 0c21bbe9e74d42c88248e815288e5909 P ca2f0c3dd5b0431a913c99ba5bfd2bd5 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:03:37.879436 15944 raft_consensus.cc:2454] T 0c21bbe9e74d42c88248e815288e5909 P ca2f0c3dd5b0431a913c99ba5bfd2bd5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb92e98ad74b42af916785483c36e382 in term 1.
I20260812 21:03:37.879439 15670 raft_consensus.cc:2454] T 0c21bbe9e74d42c88248e815288e5909 P 92601044a2eb4651b9d3bb09cbe9d5b0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb92e98ad74b42af916785483c36e382 in term 1.
I20260812 21:03:37.879608 16077 leader_election.cc:302] T 0c21bbe9e74d42c88248e815288e5909 P cb92e98ad74b42af916785483c36e382 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ca2f0c3dd5b0431a913c99ba5bfd2bd5, cb92e98ad74b42af916785483c36e382; no voters:
I20260812 21:03:37.879689 16350 raft_consensus.cc:2781] T 0c21bbe9e74d42c88248e815288e5909 P cb92e98ad74b42af916785483c36e382 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:03:37.879827 16350 raft_consensus.cc:686] T 0c21bbe9e74d42c88248e815288e5909 P cb92e98ad74b42af916785483c36e382 [term 1 LEADER]: Becoming Leader. State: Replica: cb92e98ad74b42af916785483c36e382, State: Running, Role: LEADER
I20260812 21:03:37.879925 16350 consensus_queue.cc:237] T 0c21bbe9e74d42c88248e815288e5909 P cb92e98ad74b42af916785483c36e382 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92601044a2eb4651b9d3bb09cbe9d5b0" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 33109 } } peers { permanent_uuid: "cb92e98ad74b42af916785483c36e382" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 35785 } } peers { permanent_uuid: "ca2f0c3dd5b0431a913c99ba5bfd2bd5" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 44949 } }
I20260812 21:03:37.880775 15361 catalog_manager.cc:5495] T 0c21bbe9e74d42c88248e815288e5909 P cb92e98ad74b42af916785483c36e382 reported cstate change: term changed from 0 to 1, leader changed from <none> to cb92e98ad74b42af916785483c36e382 (127.11.254.131). New cstate: current_term: 1 leader_uuid: "cb92e98ad74b42af916785483c36e382" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92601044a2eb4651b9d3bb09cbe9d5b0" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 33109 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb92e98ad74b42af916785483c36e382" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 35785 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ca2f0c3dd5b0431a913c99ba5bfd2bd5" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 44949 } health_report { overall_health: UNKNOWN } } }
Aug 12 21:03:37 dist-test-slave-vpvm krb5kdc[15282](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568616, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.190@KRBTEST.COM for kudu/127.11.254.190@KRBTEST.COM
I20260812 21:03:37.930187 16368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:37.922825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:45761 (local address 127.0.0.1:35854)
0812 21:03:37.923021 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0812 21:03:37.923033 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 21:03:37.923109 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:03:37.923254 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:03:37.923255 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:03:37.923681 (+ 426us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:03:37.923737 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:37.923741 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:37.924390 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:37.924391 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:37.925015 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:37.925018 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:37.925359 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:37.925360 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:37.925397 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:37.925937 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:03:37.925942 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:03:37.927480 (+ 1538us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:03:37.928959 (+ 1479us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:03:37.928967 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:03:37.928969 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:03:37.929182 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:03:37.929441 (+ 259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:03:37.929442 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:03:37.929443 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:03:37.929540 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:03:37.929829 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:03:37.929837 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:03:37.929887 (+ 50us) client_negotiation.cc:769] Sending connection context
0812 21:03:37.929974 (+ 87us) client_negotiation.cc:240] Negotiation successful
0812 21:03:37.930097 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":94,"threads_started":1}
I20260812 21:03:37.930266 15491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:37.922960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35854 (local address 127.11.254.190:45761)
0812 21:03:37.923040 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0812 21:03:37.923042 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:03:37.923105 (+ 63us) server_negotiation.cc:407] Connection header received
0812 21:03:37.923176 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:03:37.923177 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:03:37.923186 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:03:37.923240 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:03:37.923808 (+ 568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:37.924319 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:37.925090 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:37.925294 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:37.925349 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:37.927574 (+ 2225us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:03:37.927575 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:03:37.927575 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:03:37.927582 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:03:37.928868 (+ 1286us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:37.929288 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:37.929290 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:37.929290 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:37.929355 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:37.929599 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:37.929600 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:37.929601 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:37.929747 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:03:37.929806 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0812 21:03:37.930010 (+ 204us) server_negotiation.cc:299] Negotiation successful
0812 21:03:37.930165 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":63,"server-negotiator.queue_time_us":51}
I20260812 21:03:37.932215 15361 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.11.254.190@KRBTEST.COM'} at 127.0.0.1:35854:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260812 21:03:37.932425 15361 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 21:03:37.933835 15630 tablet_service.cc:1467] Processing CreateTablet for tablet ef87aefc414947ddaa1a07f8b5c28dfe (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=1ebbf99e536a4496ae734d4d3e35fb0f]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 21:03:37.933885 16169 tablet_service.cc:1467] Processing CreateTablet for tablet ef87aefc414947ddaa1a07f8b5c28dfe (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=1ebbf99e536a4496ae734d4d3e35fb0f]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 21:03:37.934026 15630 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ef87aefc414947ddaa1a07f8b5c28dfe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:03:37.934062 16169 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ef87aefc414947ddaa1a07f8b5c28dfe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:03:37.935168 16338 tablet_bootstrap.cc:492] T ef87aefc414947ddaa1a07f8b5c28dfe P 92601044a2eb4651b9d3bb09cbe9d5b0: Bootstrap starting.
I20260812 21:03:37.935232 16342 tablet_bootstrap.cc:492] T ef87aefc414947ddaa1a07f8b5c28dfe P cb92e98ad74b42af916785483c36e382: Bootstrap starting.
I20260812 21:03:37.935278 15906 tablet_service.cc:1467] Processing CreateTablet for tablet ef87aefc414947ddaa1a07f8b5c28dfe (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=1ebbf99e536a4496ae734d4d3e35fb0f]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 21:03:37.935415 15906 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ef87aefc414947ddaa1a07f8b5c28dfe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:03:37.935602 16338 tablet_bootstrap.cc:654] T ef87aefc414947ddaa1a07f8b5c28dfe P 92601044a2eb4651b9d3bb09cbe9d5b0: Neither blocks nor log segments found. Creating new log.
I20260812 21:03:37.935602 16342 tablet_bootstrap.cc:654] T ef87aefc414947ddaa1a07f8b5c28dfe P cb92e98ad74b42af916785483c36e382: Neither blocks nor log segments found. Creating new log.
I20260812 21:03:37.936273 16338 tablet_bootstrap.cc:492] T ef87aefc414947ddaa1a07f8b5c28dfe P 92601044a2eb4651b9d3bb09cbe9d5b0: No bootstrap required, opened a new log
I20260812 21:03:37.936329 16338 ts_tablet_manager.cc:1397] T ef87aefc414947ddaa1a07f8b5c28dfe P 92601044a2eb4651b9d3bb09cbe9d5b0: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 21:03:37.936431 16342 tablet_bootstrap.cc:492] T ef87aefc414947ddaa1a07f8b5c28dfe P cb92e98ad74b42af916785483c36e382: No bootstrap required, opened a new log
I20260812 21:03:37.936477 16342 ts_tablet_manager.cc:1397] T ef87aefc414947ddaa1a07f8b5c28dfe P cb92e98ad74b42af916785483c36e382: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260812 21:03:37.936486 16338 raft_consensus.cc:348] T ef87aefc414947ddaa1a07f8b5c28dfe P 92601044a2eb4651b9d3bb09cbe9d5b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92601044a2eb4651b9d3bb09cbe9d5b0" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 33109 } } peers { permanent_uuid: "ca2f0c3dd5b0431a913c99ba5bfd2bd5" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 44949 } } peers { permanent_uuid: "cb92e98ad74b42af916785483c36e382" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 35785 } }
I20260812 21:03:37.936570 16338 raft_consensus.cc:374] T ef87aefc414947ddaa1a07f8b5c28dfe P 92601044a2eb4651b9d3bb09cbe9d5b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:03:37.936594 16338 raft_consensus.cc:729] T ef87aefc414947ddaa1a07f8b5c28dfe P 92601044a2eb4651b9d3bb09cbe9d5b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92601044a2eb4651b9d3bb09cbe9d5b0, State: Initialized, Role: FOLLOWER
I20260812 21:03:37.936622 16342 raft_consensus.cc:348] T ef87aefc414947ddaa1a07f8b5c28dfe P cb92e98ad74b42af916785483c36e382 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92601044a2eb4651b9d3bb09cbe9d5b0" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 33109 } } peers { permanent_uuid: "ca2f0c3dd5b0431a913c99ba5bfd2bd5" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 44949 } } peers { permanent_uuid: "cb92e98ad74b42af916785483c36e382" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 35785 } }
I20260812 21:03:37.936642 16338 consensus_queue.cc:260] T ef87aefc414947ddaa1a07f8b5c28dfe P 92601044a2eb4651b9d3bb09cbe9d5b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92601044a2eb4651b9d3bb09cbe9d5b0" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 33109 } } peers { permanent_uuid: "ca2f0c3dd5b0431a913c99ba5bfd2bd5" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 44949 } } peers { permanent_uuid: "cb92e98ad74b42af916785483c36e382" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 35785 } }
I20260812 21:03:37.936688 16342 raft_consensus.cc:374] T ef87aefc414947ddaa1a07f8b5c28dfe P cb92e98ad74b42af916785483c36e382 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:03:37.936708 16342 raft_consensus.cc:729] T ef87aefc414947ddaa1a07f8b5c28dfe P cb92e98ad74b42af916785483c36e382 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb92e98ad74b42af916785483c36e382, State: Initialized, Role: FOLLOWER
I20260812 21:03:37.936697 16340 tablet_bootstrap.cc:492] T ef87aefc414947ddaa1a07f8b5c28dfe P ca2f0c3dd5b0431a913c99ba5bfd2bd5: Bootstrap starting.
I20260812 21:03:37.936733 16338 ts_tablet_manager.cc:1428] T ef87aefc414947ddaa1a07f8b5c28dfe P 92601044a2eb4651b9d3bb09cbe9d5b0: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260812 21:03:37.936746 16342 consensus_queue.cc:260] T ef87aefc414947ddaa1a07f8b5c28dfe P cb92e98ad74b42af916785483c36e382 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92601044a2eb4651b9d3bb09cbe9d5b0" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 33109 } } peers { permanent_uuid: "ca2f0c3dd5b0431a913c99ba5bfd2bd5" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 44949 } } peers { permanent_uuid: "cb92e98ad74b42af916785483c36e382" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 35785 } }
I20260812 21:03:37.936822 16342 ts_tablet_manager.cc:1428] T ef87aefc414947ddaa1a07f8b5c28dfe P cb92e98ad74b42af916785483c36e382: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260812 21:03:37.936888 16343 tablet_replica.cc:439] T ef87aefc414947ddaa1a07f8b5c28dfe P 92601044a2eb4651b9d3bb09cbe9d5b0: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92601044a2eb4651b9d3bb09cbe9d5b0" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 33109 } } peers { permanent_uuid: "ca2f0c3dd5b0431a913c99ba5bfd2bd5" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 44949 } } peers { permanent_uuid: "cb92e98ad74b42af916785483c36e382" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 35785 } } }
I20260812 21:03:37.936936 16350 tablet_replica.cc:439] T ef87aefc414947ddaa1a07f8b5c28dfe P cb92e98ad74b42af916785483c36e382: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92601044a2eb4651b9d3bb09cbe9d5b0" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 33109 } } peers { permanent_uuid: "ca2f0c3dd5b0431a913c99ba5bfd2bd5" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 44949 } } peers { permanent_uuid: "cb92e98ad74b42af916785483c36e382" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 35785 } } }
I20260812 21:03:37.936968 16343 tablet_replica.cc:442] T ef87aefc414947ddaa1a07f8b5c28dfe P 92601044a2eb4651b9d3bb09cbe9d5b0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 21:03:37.937000 16350 tablet_replica.cc:442] T ef87aefc414947ddaa1a07f8b5c28dfe P cb92e98ad74b42af916785483c36e382: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 21:03:37.937074 16340 tablet_bootstrap.cc:654] T ef87aefc414947ddaa1a07f8b5c28dfe P ca2f0c3dd5b0431a913c99ba5bfd2bd5: Neither blocks nor log segments found. Creating new log.
I20260812 21:03:37.937716 16340 tablet_bootstrap.cc:492] T ef87aefc414947ddaa1a07f8b5c28dfe P ca2f0c3dd5b0431a913c99ba5bfd2bd5: No bootstrap required, opened a new log
I20260812 21:03:37.937768 16340 ts_tablet_manager.cc:1397] T ef87aefc414947ddaa1a07f8b5c28dfe P ca2f0c3dd5b0431a913c99ba5bfd2bd5: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 21:03:37.937917 16340 raft_consensus.cc:348] T ef87aefc414947ddaa1a07f8b5c28dfe P ca2f0c3dd5b0431a913c99ba5bfd2bd5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92601044a2eb4651b9d3bb09cbe9d5b0" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 33109 } } peers { permanent_uuid: "ca2f0c3dd5b0431a913c99ba5bfd2bd5" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 44949 } } peers { permanent_uuid: "cb92e98ad74b42af916785483c36e382" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 35785 } }
I20260812 21:03:37.938009 16340 raft_consensus.cc:374] T ef87aefc414947ddaa1a07f8b5c28dfe P ca2f0c3dd5b0431a913c99ba5bfd2bd5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:03:37.938045 16340 raft_consensus.cc:729] T ef87aefc414947ddaa1a07f8b5c28dfe P ca2f0c3dd5b0431a913c99ba5bfd2bd5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca2f0c3dd5b0431a913c99ba5bfd2bd5, State: Initialized, Role: FOLLOWER
I20260812 21:03:37.938095 16340 consensus_queue.cc:260] T ef87aefc414947ddaa1a07f8b5c28dfe P ca2f0c3dd5b0431a913c99ba5bfd2bd5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92601044a2eb4651b9d3bb09cbe9d5b0" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 33109 } } peers { permanent_uuid: "ca2f0c3dd5b0431a913c99ba5bfd2bd5" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 44949 } } peers { permanent_uuid: "cb92e98ad74b42af916785483c36e382" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 35785 } }
I20260812 21:03:37.938197 16340 ts_tablet_manager.cc:1428] T ef87aefc414947ddaa1a07f8b5c28dfe P ca2f0c3dd5b0431a913c99ba5bfd2bd5: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 21:03:37.938271 16349 tablet_replica.cc:439] T ef87aefc414947ddaa1a07f8b5c28dfe P ca2f0c3dd5b0431a913c99ba5bfd2bd5: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92601044a2eb4651b9d3bb09cbe9d5b0" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 33109 } } peers { permanent_uuid: "ca2f0c3dd5b0431a913c99ba5bfd2bd5" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 44949 } } peers { permanent_uuid: "cb92e98ad74b42af916785483c36e382" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 35785 } } }
I20260812 21:03:37.938359 16349 tablet_replica.cc:442] T ef87aefc414947ddaa1a07f8b5c28dfe P ca2f0c3dd5b0431a913c99ba5bfd2bd5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 21:03:37.980443 16343 raft_consensus.cc:482] T ef87aefc414947ddaa1a07f8b5c28dfe P 92601044a2eb4651b9d3bb09cbe9d5b0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 21:03:37.980614 16343 raft_consensus.cc:504] T ef87aefc414947ddaa1a07f8b5c28dfe P 92601044a2eb4651b9d3bb09cbe9d5b0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92601044a2eb4651b9d3bb09cbe9d5b0" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 33109 } } peers { permanent_uuid: "ca2f0c3dd5b0431a913c99ba5bfd2bd5" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 44949 } } peers { permanent_uuid: "cb92e98ad74b42af916785483c36e382" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 35785 } }
I20260812 21:03:37.980971 16343 leader_election.cc:288] T ef87aefc414947ddaa1a07f8b5c28dfe P 92601044a2eb4651b9d3bb09cbe9d5b0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ca2f0c3dd5b0431a913c99ba5bfd2bd5 (127.11.254.130:44949), cb92e98ad74b42af916785483c36e382 (127.11.254.131:35785)
I20260812 21:03:37.984043 16377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:37.981114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.130:44949 (local address 127.11.254.129:40025)
0812 21:03:37.981255 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0812 21:03:37.981266 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 21:03:37.981329 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:03:37.981642 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:03:37.981643 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:03:37.981645 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:03:37.981734 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:37.981736 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:37.982403 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:37.982404 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:37.983288 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:37.983290 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:37.983826 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:37.983827 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:37.983897 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:37.983901 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 21:03:37.983941 (+ 40us) client_negotiation.cc:240] Negotiation successful
0812 21:03:37.983970 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"mutex_wait_us":31,"thread_start_us":86,"threads_started":1}
I20260812 21:03:37.984077 16335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:37.981199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.129:40025 (local address 127.11.254.130:44949)
0812 21:03:37.981255 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0812 21:03:37.981257 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:03:37.981326 (+ 69us) server_negotiation.cc:407] Connection header received
0812 21:03:37.981563 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:03:37.981564 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:03:37.981574 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:03:37.981623 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:03:37.981794 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:37.982328 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:37.983370 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:37.983726 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:37.983790 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:37.983799 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 21:03:37.983959 (+ 160us) server_negotiation.cc:299] Negotiation successful
0812 21:03:37.984002 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260812 21:03:37.984278 15944 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef87aefc414947ddaa1a07f8b5c28dfe" candidate_uuid: "92601044a2eb4651b9d3bb09cbe9d5b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca2f0c3dd5b0431a913c99ba5bfd2bd5" is_pre_election: true
I20260812 21:03:37.984354 16378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:37.981317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.131:35785 (local address 127.11.254.129:36455)
0812 21:03:37.981410 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0812 21:03:37.981421 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 21:03:37.981485 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:03:37.981665 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:03:37.981665 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:03:37.981667 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:03:37.981741 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:37.981742 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:37.982406 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:37.982407 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:37.983290 (+ 883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:37.983292 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:37.984103 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:37.984104 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:37.984174 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:37.984178 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 21:03:37.984234 (+ 56us) client_negotiation.cc:240] Negotiation successful
0812 21:03:37.984271 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":78,"mutex_wait_us":51,"thread_start_us":48,"threads_started":1}
I20260812 21:03:37.984383 15944 raft_consensus.cc:2454] T ef87aefc414947ddaa1a07f8b5c28dfe P ca2f0c3dd5b0431a913c99ba5bfd2bd5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92601044a2eb4651b9d3bb09cbe9d5b0 in term 0.
I20260812 21:03:37.984454 16379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:37.981392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.129:36455 (local address 127.11.254.131:35785)
0812 21:03:37.981514 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0812 21:03:37.981516 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:03:37.981523 (+ 7us) server_negotiation.cc:407] Connection header received
0812 21:03:37.981546 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:03:37.981546 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:03:37.981554 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:03:37.981645 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:03:37.981802 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:37.982328 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:37.983376 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:37.983735 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:37.983797 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:37.983806 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 21:03:37.984256 (+ 450us) server_negotiation.cc:299] Negotiation successful
0812 21:03:37.984309 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103,"thread_start_us":77,"threads_started":1}
I20260812 21:03:37.984575 15547 leader_election.cc:302] T ef87aefc414947ddaa1a07f8b5c28dfe P 92601044a2eb4651b9d3bb09cbe9d5b0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92601044a2eb4651b9d3bb09cbe9d5b0, ca2f0c3dd5b0431a913c99ba5bfd2bd5; no voters:
I20260812 21:03:37.984719 16343 raft_consensus.cc:2781] T ef87aefc414947ddaa1a07f8b5c28dfe P 92601044a2eb4651b9d3bb09cbe9d5b0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 21:03:37.984761 16343 raft_consensus.cc:482] T ef87aefc414947ddaa1a07f8b5c28dfe P 92601044a2eb4651b9d3bb09cbe9d5b0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 21:03:37.984781 16343 raft_consensus.cc:3037] T ef87aefc414947ddaa1a07f8b5c28dfe P 92601044a2eb4651b9d3bb09cbe9d5b0 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:03:37.984735 16205 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef87aefc414947ddaa1a07f8b5c28dfe" candidate_uuid: "92601044a2eb4651b9d3bb09cbe9d5b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb92e98ad74b42af916785483c36e382" is_pre_election: true
I20260812 21:03:37.984853 16205 raft_consensus.cc:2454] T ef87aefc414947ddaa1a07f8b5c28dfe P cb92e98ad74b42af916785483c36e382 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92601044a2eb4651b9d3bb09cbe9d5b0 in term 0.
I20260812 21:03:37.985513 16343 raft_consensus.cc:504] T ef87aefc414947ddaa1a07f8b5c28dfe P 92601044a2eb4651b9d3bb09cbe9d5b0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92601044a2eb4651b9d3bb09cbe9d5b0" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 33109 } } peers { permanent_uuid: "ca2f0c3dd5b0431a913c99ba5bfd2bd5" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 44949 } } peers { permanent_uuid: "cb92e98ad74b42af916785483c36e382" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 35785 } }
I20260812 21:03:37.985641 16343 leader_election.cc:288] T ef87aefc414947ddaa1a07f8b5c28dfe P 92601044a2eb4651b9d3bb09cbe9d5b0 [CANDIDATE]: Term 1 election: Requested vote from peers ca2f0c3dd5b0431a913c99ba5bfd2bd5 (127.11.254.130:44949), cb92e98ad74b42af916785483c36e382 (127.11.254.131:35785)
I20260812 21:03:37.985780 15944 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef87aefc414947ddaa1a07f8b5c28dfe" candidate_uuid: "92601044a2eb4651b9d3bb09cbe9d5b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca2f0c3dd5b0431a913c99ba5bfd2bd5"
I20260812 21:03:37.985805 16205 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef87aefc414947ddaa1a07f8b5c28dfe" candidate_uuid: "92601044a2eb4651b9d3bb09cbe9d5b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb92e98ad74b42af916785483c36e382"
I20260812 21:03:37.985862 16205 raft_consensus.cc:3037] T ef87aefc414947ddaa1a07f8b5c28dfe P cb92e98ad74b42af916785483c36e382 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:03:37.985862 15944 raft_consensus.cc:3037] T ef87aefc414947ddaa1a07f8b5c28dfe P ca2f0c3dd5b0431a913c99ba5bfd2bd5 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:03:37.986492 16205 raft_consensus.cc:2454] T ef87aefc414947ddaa1a07f8b5c28dfe P cb92e98ad74b42af916785483c36e382 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92601044a2eb4651b9d3bb09cbe9d5b0 in term 1.
I20260812 21:03:37.986501 15944 raft_consensus.cc:2454] T ef87aefc414947ddaa1a07f8b5c28dfe P ca2f0c3dd5b0431a913c99ba5bfd2bd5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92601044a2eb4651b9d3bb09cbe9d5b0 in term 1.
I20260812 21:03:37.986650 15547 leader_election.cc:302] T ef87aefc414947ddaa1a07f8b5c28dfe P 92601044a2eb4651b9d3bb09cbe9d5b0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92601044a2eb4651b9d3bb09cbe9d5b0, ca2f0c3dd5b0431a913c99ba5bfd2bd5; no voters:
I20260812 21:03:37.986770 16343 raft_consensus.cc:2781] T ef87aefc414947ddaa1a07f8b5c28dfe P 92601044a2eb4651b9d3bb09cbe9d5b0 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:03:37.986907 16343 raft_consensus.cc:686] T ef87aefc414947ddaa1a07f8b5c28dfe P 92601044a2eb4651b9d3bb09cbe9d5b0 [term 1 LEADER]: Becoming Leader. State: Replica: 92601044a2eb4651b9d3bb09cbe9d5b0, State: Running, Role: LEADER
I20260812 21:03:37.986997 16343 consensus_queue.cc:237] T ef87aefc414947ddaa1a07f8b5c28dfe P 92601044a2eb4651b9d3bb09cbe9d5b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92601044a2eb4651b9d3bb09cbe9d5b0" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 33109 } } peers { permanent_uuid: "ca2f0c3dd5b0431a913c99ba5bfd2bd5" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 44949 } } peers { permanent_uuid: "cb92e98ad74b42af916785483c36e382" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 35785 } }
I20260812 21:03:37.987411 16380 tablet_replica.cc:439] T ef87aefc414947ddaa1a07f8b5c28dfe P 92601044a2eb4651b9d3bb09cbe9d5b0: TxnStatusTablet state changed. Reason: New leader 92601044a2eb4651b9d3bb09cbe9d5b0. Latest consensus state: current_term: 1 leader_uuid: "92601044a2eb4651b9d3bb09cbe9d5b0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92601044a2eb4651b9d3bb09cbe9d5b0" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 33109 } } peers { permanent_uuid: "ca2f0c3dd5b0431a913c99ba5bfd2bd5" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 44949 } } peers { permanent_uuid: "cb92e98ad74b42af916785483c36e382" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 35785 } } }
I20260812 21:03:37.987550 16380 tablet_replica.cc:442] T ef87aefc414947ddaa1a07f8b5c28dfe P 92601044a2eb4651b9d3bb09cbe9d5b0: This TxnStatusTablet replica's current role is: LEADER
I20260812 21:03:37.987783 15357 catalog_manager.cc:5495] T ef87aefc414947ddaa1a07f8b5c28dfe P 92601044a2eb4651b9d3bb09cbe9d5b0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 92601044a2eb4651b9d3bb09cbe9d5b0 (127.11.254.129). New cstate: current_term: 1 leader_uuid: "92601044a2eb4651b9d3bb09cbe9d5b0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92601044a2eb4651b9d3bb09cbe9d5b0" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 33109 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ca2f0c3dd5b0431a913c99ba5bfd2bd5" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 44949 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb92e98ad74b42af916785483c36e382" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 35785 } health_report { overall_health: UNKNOWN } } }
I20260812 21:03:37.988063 16385 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260812 21:03:37.988165 16385 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260812 21:03:37.998212 16368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:37.995096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.129:33109 (local address 127.0.0.1:40918)
0812 21:03:37.995179 (+ 83us) negotiation.cc:107] Waiting for socket to connect
0812 21:03:37.995191 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 21:03:37.995241 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:03:37.995437 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:03:37.995438 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:03:37.995439 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 21:03:37.995498 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:37.995501 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:37.996153 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:37.996155 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:37.997099 (+ 944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:37.997103 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:37.997608 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:37.997610 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:37.997692 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:37.997714 (+ 22us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 21:03:37.998014 (+ 300us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 21:03:37.998024 (+ 10us) client_negotiation.cc:769] Sending connection context
0812 21:03:37.998076 (+ 52us) client_negotiation.cc:240] Negotiation successful
0812 21:03:37.998133 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260812 21:03:37.998212 16330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:37.995147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40918 (local address 127.11.254.129:33109)
0812 21:03:37.995213 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0812 21:03:37.995215 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:03:37.995224 (+ 9us) server_negotiation.cc:407] Connection header received
0812 21:03:37.995326 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:03:37.995327 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:03:37.995348 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:03:37.995420 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 21:03:37.995571 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:37.996066 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:37.997220 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:37.997521 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:37.997589 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:37.997806 (+ 217us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 21:03:37.997920 (+ 114us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 21:03:37.998042 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0812 21:03:37.998083 (+ 41us) server_negotiation.cc:299] Negotiation successful
0812 21:03:37.998129 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260812 21:03:38.000113 16205 raft_consensus.cc:1261] T ef87aefc414947ddaa1a07f8b5c28dfe P cb92e98ad74b42af916785483c36e382 [term 1 FOLLOWER]: Refusing update from remote peer 92601044a2eb4651b9d3bb09cbe9d5b0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 21:03:38.000443 15944 raft_consensus.cc:1261] T ef87aefc414947ddaa1a07f8b5c28dfe P ca2f0c3dd5b0431a913c99ba5bfd2bd5 [term 1 FOLLOWER]: Refusing update from remote peer 92601044a2eb4651b9d3bb09cbe9d5b0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 21:03:38.000440 16343 consensus_queue.cc:1035] T ef87aefc414947ddaa1a07f8b5c28dfe P 92601044a2eb4651b9d3bb09cbe9d5b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb92e98ad74b42af916785483c36e382" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 35785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 21:03:38.000810 16343 consensus_queue.cc:1035] T ef87aefc414947ddaa1a07f8b5c28dfe P 92601044a2eb4651b9d3bb09cbe9d5b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca2f0c3dd5b0431a913c99ba5bfd2bd5" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 44949 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 21:03:38.001833 16350 tablet_replica.cc:439] T ef87aefc414947ddaa1a07f8b5c28dfe P cb92e98ad74b42af916785483c36e382: TxnStatusTablet state changed. Reason: New leader 92601044a2eb4651b9d3bb09cbe9d5b0. Latest consensus state: current_term: 1 leader_uuid: "92601044a2eb4651b9d3bb09cbe9d5b0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92601044a2eb4651b9d3bb09cbe9d5b0" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 33109 } } peers { permanent_uuid: "ca2f0c3dd5b0431a913c99ba5bfd2bd5" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 44949 } } peers { permanent_uuid: "cb92e98ad74b42af916785483c36e382" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 35785 } } }
I20260812 21:03:38.001987 16349 tablet_replica.cc:439] T ef87aefc414947ddaa1a07f8b5c28dfe P ca2f0c3dd5b0431a913c99ba5bfd2bd5: TxnStatusTablet state changed. Reason: New leader 92601044a2eb4651b9d3bb09cbe9d5b0. Latest consensus state: current_term: 1 leader_uuid: "92601044a2eb4651b9d3bb09cbe9d5b0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92601044a2eb4651b9d3bb09cbe9d5b0" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 33109 } } peers { permanent_uuid: "ca2f0c3dd5b0431a913c99ba5bfd2bd5" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 44949 } } peers { permanent_uuid: "cb92e98ad74b42af916785483c36e382" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 35785 } } }
I20260812 21:03:38.002086 16349 tablet_replica.cc:442] T ef87aefc414947ddaa1a07f8b5c28dfe P ca2f0c3dd5b0431a913c99ba5bfd2bd5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 21:03:38.002862 16350 tablet_replica.cc:442] T ef87aefc414947ddaa1a07f8b5c28dfe P cb92e98ad74b42af916785483c36e382: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 21:03:38.002921 16380 tablet_replica.cc:439] T ef87aefc414947ddaa1a07f8b5c28dfe P 92601044a2eb4651b9d3bb09cbe9d5b0: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "92601044a2eb4651b9d3bb09cbe9d5b0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92601044a2eb4651b9d3bb09cbe9d5b0" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 33109 } } peers { permanent_uuid: "ca2f0c3dd5b0431a913c99ba5bfd2bd5" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 44949 } } peers { permanent_uuid: "cb92e98ad74b42af916785483c36e382" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 35785 } } }
I20260812 21:03:38.003008 16380 tablet_replica.cc:442] T ef87aefc414947ddaa1a07f8b5c28dfe P 92601044a2eb4651b9d3bb09cbe9d5b0: This TxnStatusTablet replica's current role is: LEADER
I20260812 21:03:38.003494 16349 tablet_replica.cc:439] T ef87aefc414947ddaa1a07f8b5c28dfe P ca2f0c3dd5b0431a913c99ba5bfd2bd5: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "92601044a2eb4651b9d3bb09cbe9d5b0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92601044a2eb4651b9d3bb09cbe9d5b0" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 33109 } } peers { permanent_uuid: "ca2f0c3dd5b0431a913c99ba5bfd2bd5" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 44949 } } peers { permanent_uuid: "cb92e98ad74b42af916785483c36e382" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 35785 } } }
I20260812 21:03:38.003577 16349 tablet_replica.cc:442] T ef87aefc414947ddaa1a07f8b5c28dfe P ca2f0c3dd5b0431a913c99ba5bfd2bd5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 21:03:38.004501 16343 tablet_replica.cc:439] T ef87aefc414947ddaa1a07f8b5c28dfe P 92601044a2eb4651b9d3bb09cbe9d5b0: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "92601044a2eb4651b9d3bb09cbe9d5b0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92601044a2eb4651b9d3bb09cbe9d5b0" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 33109 } } peers { permanent_uuid: "ca2f0c3dd5b0431a913c99ba5bfd2bd5" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 44949 } } peers { permanent_uuid: "cb92e98ad74b42af916785483c36e382" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 35785 } } }
I20260812 21:03:38.004586 16343 tablet_replica.cc:442] T ef87aefc414947ddaa1a07f8b5c28dfe P 92601044a2eb4651b9d3bb09cbe9d5b0: This TxnStatusTablet replica's current role is: LEADER
I20260812 21:03:38.004860 16392 mvcc.cc:205] Tried to move back new op lower bound from 7317785059325747200 to 7317785059275309056. Current Snapshot: MvccSnapshot[applied={T|T < 7317785059325747200 or (T in {7317785059325747200})}]
I20260812 21:03:38.004913 16350 tablet_replica.cc:439] T ef87aefc414947ddaa1a07f8b5c28dfe P cb92e98ad74b42af916785483c36e382: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "92601044a2eb4651b9d3bb09cbe9d5b0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92601044a2eb4651b9d3bb09cbe9d5b0" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 33109 } } peers { permanent_uuid: "ca2f0c3dd5b0431a913c99ba5bfd2bd5" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 44949 } } peers { permanent_uuid: "cb92e98ad74b42af916785483c36e382" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 35785 } } }
I20260812 21:03:38.005000 16350 tablet_replica.cc:442] T ef87aefc414947ddaa1a07f8b5c28dfe P cb92e98ad74b42af916785483c36e382: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 21:03:38.005376 16391 mvcc.cc:205] Tried to move back new op lower bound from 7317785059325747200 to 7317785059275309056. Current Snapshot: MvccSnapshot[applied={T|T < 7317785059325747200}]
I20260812 21:03:38.008558 16379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:38.006053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57416 (local address 127.11.254.131:35785)
0812 21:03:38.006095 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0812 21:03:38.006097 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:03:38.006252 (+ 155us) server_negotiation.cc:407] Connection header received
0812 21:03:38.006330 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:03:38.006331 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:03:38.006349 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:03:38.006418 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 21:03:38.006550 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:38.007033 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:38.007800 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:38.008017 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:38.008079 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:38.008235 (+ 156us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 21:03:38.008334 (+ 99us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 21:03:38.008404 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 21:03:38.008462 (+ 58us) server_negotiation.cc:299] Negotiation successful
0812 21:03:38.008485 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260812 21:03:38.014137 16330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:38.011411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.131:58161 (local address 127.11.254.129:33109)
0812 21:03:38.011490 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0812 21:03:38.011492 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:03:38.011555 (+ 63us) server_negotiation.cc:407] Connection header received
0812 21:03:38.011631 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:03:38.011632 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:03:38.011640 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:03:38.011684 (+ 44us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 21:03:38.011856 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:38.012345 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:38.013454 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:38.013645 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:38.013690 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:38.013895 (+ 205us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 21:03:38.013922 (+ 27us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 21:03:38.013988 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0812 21:03:38.014049 (+ 61us) server_negotiation.cc:299] Negotiation successful
0812 21:03:38.014069 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 21:03:38.014202 16410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:38.011334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.129:33109 (local address 127.11.254.131:58161)
0812 21:03:38.011496 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0812 21:03:38.011508 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 21:03:38.011573 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:03:38.011688 (+ 115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:03:38.011688 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:03:38.011690 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 21:03:38.011762 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:38.011764 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:38.012414 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:38.012415 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:38.013365 (+ 950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:38.013368 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:38.013706 (+ 338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:38.013707 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:38.013780 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:38.013798 (+ 18us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 21:03:38.013993 (+ 195us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 21:03:38.013995 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 21:03:38.014047 (+ 52us) client_negotiation.cc:240] Negotiation successful
0812 21:03:38.014096 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":83,"threads_started":1}
I20260812 21:03:38.017195 15670 raft_consensus.cc:1261] T 0c21bbe9e74d42c88248e815288e5909 P 92601044a2eb4651b9d3bb09cbe9d5b0 [term 1 FOLLOWER]: Refusing update from remote peer cb92e98ad74b42af916785483c36e382: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 21:03:38.017393 15944 raft_consensus.cc:1261] T 0c21bbe9e74d42c88248e815288e5909 P ca2f0c3dd5b0431a913c99ba5bfd2bd5 [term 1 FOLLOWER]: Refusing update from remote peer cb92e98ad74b42af916785483c36e382: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 21:03:38.017603 16350 consensus_queue.cc:1035] T 0c21bbe9e74d42c88248e815288e5909 P cb92e98ad74b42af916785483c36e382 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca2f0c3dd5b0431a913c99ba5bfd2bd5" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 44949 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 21:03:38.017778 16354 consensus_queue.cc:1035] T 0c21bbe9e74d42c88248e815288e5909 P cb92e98ad74b42af916785483c36e382 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92601044a2eb4651b9d3bb09cbe9d5b0" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 33109 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 21:03:38.019269 16390 mvcc.cc:205] Tried to move back new op lower bound from 7317785059396222976 to 7317785058836594688. Current Snapshot: MvccSnapshot[applied={T|T < 7317785059396222976}]
I20260812 21:03:38.026188 16379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:38.023686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.129:43445 (local address 127.11.254.131:35785)
0812 21:03:38.023805 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0812 21:03:38.023807 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:03:38.023881 (+ 74us) server_negotiation.cc:407] Connection header received
0812 21:03:38.023939 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:03:38.023940 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:03:38.023951 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:03:38.024015 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 21:03:38.024139 (+ 124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:38.024609 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:38.025516 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:38.025729 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:38.025783 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:38.025930 (+ 147us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 21:03:38.025992 (+ 62us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 21:03:38.026035 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0812 21:03:38.026087 (+ 52us) server_negotiation.cc:299] Negotiation successful
0812 21:03:38.026112 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260812 21:03:38.026223 16418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:38.023662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.131:35785 (local address 127.11.254.129:43445)
0812 21:03:38.023809 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0812 21:03:38.023819 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 21:03:38.023893 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:03:38.024010 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:03:38.024010 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:03:38.024011 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 21:03:38.024066 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:38.024067 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:38.024686 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:38.024687 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:38.025455 (+ 768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:38.025458 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:38.025787 (+ 329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:38.025788 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:38.025850 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:38.025866 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 21:03:38.026035 (+ 169us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 21:03:38.026037 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 21:03:38.026084 (+ 47us) client_negotiation.cc:240] Negotiation successful
0812 21:03:38.026133 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":80,"threads_started":1}
I20260812 21:03:38.036458 15357 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35844:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 21:03:38.036609 15357 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35844:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 21:03:38.037487 15357 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f41525e3e1134a4887fe0909de2f21f6: Sending DeleteTablet for 3 replicas of tablet 0c21bbe9e74d42c88248e815288e5909
I20260812 21:03:38.037909 15630 tablet_service.cc:1514] Processing DeleteTablet for tablet 0c21bbe9e74d42c88248e815288e5909 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:03:38 UTC) from {username='kudu', principal='kudu/127.11.254.190@KRBTEST.COM'} at 127.0.0.1:40914
I20260812 21:03:38.037987 15906 tablet_service.cc:1514] Processing DeleteTablet for tablet 0c21bbe9e74d42c88248e815288e5909 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:03:38 UTC) from {username='kudu', principal='kudu/127.11.254.190@KRBTEST.COM'} at 127.0.0.1:53566
I20260812 21:03:38.038276 16169 tablet_service.cc:1514] Processing DeleteTablet for tablet 0c21bbe9e74d42c88248e815288e5909 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:03:38 UTC) from {username='kudu', principal='kudu/127.11.254.190@KRBTEST.COM'} at 127.0.0.1:57404
I20260812 21:03:38.038385 16431 tablet_replica.cc:330] T 0c21bbe9e74d42c88248e815288e5909 P ca2f0c3dd5b0431a913c99ba5bfd2bd5: stopping tablet replica
I20260812 21:03:38.038542 16431 raft_consensus.cc:2229] T 0c21bbe9e74d42c88248e815288e5909 P ca2f0c3dd5b0431a913c99ba5bfd2bd5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 21:03:38.038654 16431 raft_consensus.cc:2258] T 0c21bbe9e74d42c88248e815288e5909 P ca2f0c3dd5b0431a913c99ba5bfd2bd5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 21:03:38.038774 15357 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35844:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 21:03:38.038941 15357 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 21:03:38.039640 16431 ts_tablet_manager.cc:1905] T 0c21bbe9e74d42c88248e815288e5909 P ca2f0c3dd5b0431a913c99ba5bfd2bd5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 21:03:38.040396 15630 tablet_service.cc:1467] Processing CreateTablet for tablet 0438574006eb490c80df916b1abecca3 (DEFAULT_TABLE table=test-table [id=026bd7b41f4f4a509ae967322d644e74]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:03:38.040530 15630 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0438574006eb490c80df916b1abecca3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:03:38.041796 16433 tablet_replica.cc:330] T 0c21bbe9e74d42c88248e815288e5909 P cb92e98ad74b42af916785483c36e382: stopping tablet replica
I20260812 21:03:38.041910 16433 raft_consensus.cc:2229] T 0c21bbe9e74d42c88248e815288e5909 P cb92e98ad74b42af916785483c36e382 [term 1 LEADER]: Raft consensus shutting down.
I20260812 21:03:38.041939 16338 tablet_bootstrap.cc:492] T 0438574006eb490c80df916b1abecca3 P 92601044a2eb4651b9d3bb09cbe9d5b0: Bootstrap starting.
I20260812 21:03:38.042083 16430 tablet_replica.cc:330] T 0c21bbe9e74d42c88248e815288e5909 P 92601044a2eb4651b9d3bb09cbe9d5b0: stopping tablet replica
I20260812 21:03:38.042083 16433 raft_consensus.cc:2258] T 0c21bbe9e74d42c88248e815288e5909 P cb92e98ad74b42af916785483c36e382 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 21:03:38.042397 16338 tablet_bootstrap.cc:654] T 0438574006eb490c80df916b1abecca3 P 92601044a2eb4651b9d3bb09cbe9d5b0: Neither blocks nor log segments found. Creating new log.
I20260812 21:03:38.042493 16433 ts_tablet_manager.cc:1905] T 0c21bbe9e74d42c88248e815288e5909 P cb92e98ad74b42af916785483c36e382: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 21:03:38.042853 15906 tablet_service.cc:1467] Processing CreateTablet for tablet 0438574006eb490c80df916b1abecca3 (DEFAULT_TABLE table=test-table [id=026bd7b41f4f4a509ae967322d644e74]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:03:38.043231 16169 tablet_service.cc:1467] Processing CreateTablet for tablet 0438574006eb490c80df916b1abecca3 (DEFAULT_TABLE table=test-table [id=026bd7b41f4f4a509ae967322d644e74]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:03:38.043354 16169 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0438574006eb490c80df916b1abecca3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:03:38.043498 16431 ts_tablet_manager.cc:1918] T 0c21bbe9e74d42c88248e815288e5909 P ca2f0c3dd5b0431a913c99ba5bfd2bd5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 21:03:38.043612 16431 log.cc:1196] T 0c21bbe9e74d42c88248e815288e5909 P ca2f0c3dd5b0431a913c99ba5bfd2bd5: Deleting WAL directory at /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568607354921-12282-0/minicluster-data/ts-1/wal/wals/0c21bbe9e74d42c88248e815288e5909
I20260812 21:03:38.043948 16431 ts_tablet_manager.cc:1939] T 0c21bbe9e74d42c88248e815288e5909 P ca2f0c3dd5b0431a913c99ba5bfd2bd5: Deleting consensus metadata
I20260812 21:03:38.044622 16342 tablet_bootstrap.cc:492] T 0438574006eb490c80df916b1abecca3 P cb92e98ad74b42af916785483c36e382: Bootstrap starting.
I20260812 21:03:38.044723 15906 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0438574006eb490c80df916b1abecca3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:03:38.045053 16342 tablet_bootstrap.cc:654] T 0438574006eb490c80df916b1abecca3 P cb92e98ad74b42af916785483c36e382: Neither blocks nor log segments found. Creating new log.
I20260812 21:03:38.046000 16340 tablet_bootstrap.cc:492] T 0438574006eb490c80df916b1abecca3 P ca2f0c3dd5b0431a913c99ba5bfd2bd5: Bootstrap starting.
I20260812 21:03:38.046396 16340 tablet_bootstrap.cc:654] T 0438574006eb490c80df916b1abecca3 P ca2f0c3dd5b0431a913c99ba5bfd2bd5: Neither blocks nor log segments found. Creating new log.
I20260812 21:03:38.047041 16340 tablet_bootstrap.cc:492] T 0438574006eb490c80df916b1abecca3 P ca2f0c3dd5b0431a913c99ba5bfd2bd5: No bootstrap required, opened a new log
I20260812 21:03:38.047092 16340 ts_tablet_manager.cc:1397] T 0438574006eb490c80df916b1abecca3 P ca2f0c3dd5b0431a913c99ba5bfd2bd5: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 21:03:38.047242 16340 raft_consensus.cc:348] T 0438574006eb490c80df916b1abecca3 P ca2f0c3dd5b0431a913c99ba5bfd2bd5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92601044a2eb4651b9d3bb09cbe9d5b0" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 33109 } } peers { permanent_uuid: "ca2f0c3dd5b0431a913c99ba5bfd2bd5" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 44949 } } peers { permanent_uuid: "cb92e98ad74b42af916785483c36e382" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 35785 } }
I20260812 21:03:38.047317 16340 raft_consensus.cc:374] T 0438574006eb490c80df916b1abecca3 P ca2f0c3dd5b0431a913c99ba5bfd2bd5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:03:38.047336 16340 raft_consensus.cc:729] T 0438574006eb490c80df916b1abecca3 P ca2f0c3dd5b0431a913c99ba5bfd2bd5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca2f0c3dd5b0431a913c99ba5bfd2bd5, State: Initialized, Role: FOLLOWER
I20260812 21:03:38.047379 16340 consensus_queue.cc:260] T 0438574006eb490c80df916b1abecca3 P ca2f0c3dd5b0431a913c99ba5bfd2bd5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92601044a2eb4651b9d3bb09cbe9d5b0" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 33109 } } peers { permanent_uuid: "ca2f0c3dd5b0431a913c99ba5bfd2bd5" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 44949 } } peers { permanent_uuid: "cb92e98ad74b42af916785483c36e382" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 35785 } }
I20260812 21:03:38.047459 16340 ts_tablet_manager.cc:1428] T 0438574006eb490c80df916b1abecca3 P ca2f0c3dd5b0431a913c99ba5bfd2bd5: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 21:03:38.047468 15338 catalog_manager.cc:4841] TS ca2f0c3dd5b0431a913c99ba5bfd2bd5 (127.11.254.130:44949): tablet 0c21bbe9e74d42c88248e815288e5909 (table test-table [id=bb10a9699d364b92ac606dccb3272709]) successfully deleted
I20260812 21:03:38.049026 16433 ts_tablet_manager.cc:1918] T 0c21bbe9e74d42c88248e815288e5909 P cb92e98ad74b42af916785483c36e382: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 21:03:38.049113 16433 log.cc:1196] T 0c21bbe9e74d42c88248e815288e5909 P cb92e98ad74b42af916785483c36e382: Deleting WAL directory at /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568607354921-12282-0/minicluster-data/ts-2/wal/wals/0c21bbe9e74d42c88248e815288e5909
I20260812 21:03:38.049386 16433 ts_tablet_manager.cc:1939] T 0c21bbe9e74d42c88248e815288e5909 P cb92e98ad74b42af916785483c36e382: Deleting consensus metadata
I20260812 21:03:38.049780 15338 catalog_manager.cc:4841] TS cb92e98ad74b42af916785483c36e382 (127.11.254.131:35785): tablet 0c21bbe9e74d42c88248e815288e5909 (table test-table [id=bb10a9699d364b92ac606dccb3272709]) successfully deleted
I20260812 21:03:38.050180 16338 tablet_bootstrap.cc:492] T 0438574006eb490c80df916b1abecca3 P 92601044a2eb4651b9d3bb09cbe9d5b0: No bootstrap required, opened a new log
I20260812 21:03:38.050239 16338 ts_tablet_manager.cc:1397] T 0438574006eb490c80df916b1abecca3 P 92601044a2eb4651b9d3bb09cbe9d5b0: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.001s
I20260812 21:03:38.050338 16342 tablet_bootstrap.cc:492] T 0438574006eb490c80df916b1abecca3 P cb92e98ad74b42af916785483c36e382: No bootstrap required, opened a new log
I20260812 21:03:38.050382 16342 ts_tablet_manager.cc:1397] T 0438574006eb490c80df916b1abecca3 P cb92e98ad74b42af916785483c36e382: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.001s
I20260812 21:03:38.050392 16338 raft_consensus.cc:348] T 0438574006eb490c80df916b1abecca3 P 92601044a2eb4651b9d3bb09cbe9d5b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92601044a2eb4651b9d3bb09cbe9d5b0" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 33109 } } peers { permanent_uuid: "ca2f0c3dd5b0431a913c99ba5bfd2bd5" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 44949 } } peers { permanent_uuid: "cb92e98ad74b42af916785483c36e382" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 35785 } }
I20260812 21:03:38.050465 16338 raft_consensus.cc:374] T 0438574006eb490c80df916b1abecca3 P 92601044a2eb4651b9d3bb09cbe9d5b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:03:38.050484 16338 raft_consensus.cc:729] T 0438574006eb490c80df916b1abecca3 P 92601044a2eb4651b9d3bb09cbe9d5b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92601044a2eb4651b9d3bb09cbe9d5b0, State: Initialized, Role: FOLLOWER
I20260812 21:03:38.050525 16338 consensus_queue.cc:260] T 0438574006eb490c80df916b1abecca3 P 92601044a2eb4651b9d3bb09cbe9d5b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92601044a2eb4651b9d3bb09cbe9d5b0" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 33109 } } peers { permanent_uuid: "ca2f0c3dd5b0431a913c99ba5bfd2bd5" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 44949 } } peers { permanent_uuid: "cb92e98ad74b42af916785483c36e382" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 35785 } }
I20260812 21:03:38.050532 16342 raft_consensus.cc:348] T 0438574006eb490c80df916b1abecca3 P cb92e98ad74b42af916785483c36e382 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92601044a2eb4651b9d3bb09cbe9d5b0" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 33109 } } peers { permanent_uuid: "ca2f0c3dd5b0431a913c99ba5bfd2bd5" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 44949 } } peers { permanent_uuid: "cb92e98ad74b42af916785483c36e382" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 35785 } }
I20260812 21:03:38.050595 16342 raft_consensus.cc:374] T 0438574006eb490c80df916b1abecca3 P cb92e98ad74b42af916785483c36e382 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:03:38.050614 16342 raft_consensus.cc:729] T 0438574006eb490c80df916b1abecca3 P cb92e98ad74b42af916785483c36e382 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb92e98ad74b42af916785483c36e382, State: Initialized, Role: FOLLOWER
I20260812 21:03:38.050634 16338 ts_tablet_manager.cc:1428] T 0438574006eb490c80df916b1abecca3 P 92601044a2eb4651b9d3bb09cbe9d5b0: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 21:03:38.050655 16342 consensus_queue.cc:260] T 0438574006eb490c80df916b1abecca3 P cb92e98ad74b42af916785483c36e382 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92601044a2eb4651b9d3bb09cbe9d5b0" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 33109 } } peers { permanent_uuid: "ca2f0c3dd5b0431a913c99ba5bfd2bd5" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 44949 } } peers { permanent_uuid: "cb92e98ad74b42af916785483c36e382" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 35785 } }
I20260812 21:03:38.050719 16430 raft_consensus.cc:2229] T 0c21bbe9e74d42c88248e815288e5909 P 92601044a2eb4651b9d3bb09cbe9d5b0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 21:03:38.050740 16342 ts_tablet_manager.cc:1428] T 0438574006eb490c80df916b1abecca3 P cb92e98ad74b42af916785483c36e382: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 21:03:38.050792 16430 raft_consensus.cc:2258] T 0c21bbe9e74d42c88248e815288e5909 P 92601044a2eb4651b9d3bb09cbe9d5b0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 21:03:38.051976 16430 ts_tablet_manager.cc:1905] T 0c21bbe9e74d42c88248e815288e5909 P 92601044a2eb4651b9d3bb09cbe9d5b0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 21:03:38.054018 16430 ts_tablet_manager.cc:1918] T 0c21bbe9e74d42c88248e815288e5909 P 92601044a2eb4651b9d3bb09cbe9d5b0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 21:03:38.054101 16430 log.cc:1196] T 0c21bbe9e74d42c88248e815288e5909 P 92601044a2eb4651b9d3bb09cbe9d5b0: Deleting WAL directory at /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568607354921-12282-0/minicluster-data/ts-0/wal/wals/0c21bbe9e74d42c88248e815288e5909
I20260812 21:03:38.054384 16430 ts_tablet_manager.cc:1939] T 0c21bbe9e74d42c88248e815288e5909 P 92601044a2eb4651b9d3bb09cbe9d5b0: Deleting consensus metadata
I20260812 21:03:38.054860 15334 catalog_manager.cc:4841] TS 92601044a2eb4651b9d3bb09cbe9d5b0 (127.11.254.129:33109): tablet 0c21bbe9e74d42c88248e815288e5909 (table test-table [id=bb10a9699d364b92ac606dccb3272709]) successfully deleted
I20260812 21:03:38.070278 16425 raft_consensus.cc:482] T 0438574006eb490c80df916b1abecca3 P cb92e98ad74b42af916785483c36e382 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 21:03:38.070405 16425 raft_consensus.cc:504] T 0438574006eb490c80df916b1abecca3 P cb92e98ad74b42af916785483c36e382 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92601044a2eb4651b9d3bb09cbe9d5b0" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 33109 } } peers { permanent_uuid: "ca2f0c3dd5b0431a913c99ba5bfd2bd5" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 44949 } } peers { permanent_uuid: "cb92e98ad74b42af916785483c36e382" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 35785 } }
I20260812 21:03:38.070616 16425 leader_election.cc:288] T 0438574006eb490c80df916b1abecca3 P cb92e98ad74b42af916785483c36e382 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 92601044a2eb4651b9d3bb09cbe9d5b0 (127.11.254.129:33109), ca2f0c3dd5b0431a913c99ba5bfd2bd5 (127.11.254.130:44949)
I20260812 21:03:38.070827 15944 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0438574006eb490c80df916b1abecca3" candidate_uuid: "cb92e98ad74b42af916785483c36e382" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca2f0c3dd5b0431a913c99ba5bfd2bd5" is_pre_election: true
I20260812 21:03:38.070941 15944 raft_consensus.cc:2454] T 0438574006eb490c80df916b1abecca3 P ca2f0c3dd5b0431a913c99ba5bfd2bd5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb92e98ad74b42af916785483c36e382 in term 0.
I20260812 21:03:38.071056 15670 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0438574006eb490c80df916b1abecca3" candidate_uuid: "cb92e98ad74b42af916785483c36e382" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92601044a2eb4651b9d3bb09cbe9d5b0" is_pre_election: true
I20260812 21:03:38.071142 15670 raft_consensus.cc:2454] T 0438574006eb490c80df916b1abecca3 P 92601044a2eb4651b9d3bb09cbe9d5b0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb92e98ad74b42af916785483c36e382 in term 0.
I20260812 21:03:38.071309 16077 leader_election.cc:302] T 0438574006eb490c80df916b1abecca3 P cb92e98ad74b42af916785483c36e382 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ca2f0c3dd5b0431a913c99ba5bfd2bd5, cb92e98ad74b42af916785483c36e382; no voters:
I20260812 21:03:38.071412 16425 raft_consensus.cc:2781] T 0438574006eb490c80df916b1abecca3 P cb92e98ad74b42af916785483c36e382 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 21:03:38.071462 16425 raft_consensus.cc:482] T 0438574006eb490c80df916b1abecca3 P cb92e98ad74b42af916785483c36e382 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 21:03:38.071485 16425 raft_consensus.cc:3037] T 0438574006eb490c80df916b1abecca3 P cb92e98ad74b42af916785483c36e382 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:03:38.072432 16425 raft_consensus.cc:504] T 0438574006eb490c80df916b1abecca3 P cb92e98ad74b42af916785483c36e382 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92601044a2eb4651b9d3bb09cbe9d5b0" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 33109 } } peers { permanent_uuid: "ca2f0c3dd5b0431a913c99ba5bfd2bd5" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 44949 } } peers { permanent_uuid: "cb92e98ad74b42af916785483c36e382" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 35785 } }
I20260812 21:03:38.072579 16425 leader_election.cc:288] T 0438574006eb490c80df916b1abecca3 P cb92e98ad74b42af916785483c36e382 [CANDIDATE]: Term 1 election: Requested vote from peers 92601044a2eb4651b9d3bb09cbe9d5b0 (127.11.254.129:33109), ca2f0c3dd5b0431a913c99ba5bfd2bd5 (127.11.254.130:44949)
I20260812 21:03:38.072770 15670 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0438574006eb490c80df916b1abecca3" candidate_uuid: "cb92e98ad74b42af916785483c36e382" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92601044a2eb4651b9d3bb09cbe9d5b0"
I20260812 21:03:38.072819 15944 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0438574006eb490c80df916b1abecca3" candidate_uuid: "cb92e98ad74b42af916785483c36e382" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca2f0c3dd5b0431a913c99ba5bfd2bd5"
I20260812 21:03:38.072875 15670 raft_consensus.cc:3037] T 0438574006eb490c80df916b1abecca3 P 92601044a2eb4651b9d3bb09cbe9d5b0 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:03:38.072899 15944 raft_consensus.cc:3037] T 0438574006eb490c80df916b1abecca3 P ca2f0c3dd5b0431a913c99ba5bfd2bd5 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:03:38.073817 15670 raft_consensus.cc:2454] T 0438574006eb490c80df916b1abecca3 P 92601044a2eb4651b9d3bb09cbe9d5b0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb92e98ad74b42af916785483c36e382 in term 1.
I20260812 21:03:38.073822 15944 raft_consensus.cc:2454] T 0438574006eb490c80df916b1abecca3 P ca2f0c3dd5b0431a913c99ba5bfd2bd5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb92e98ad74b42af916785483c36e382 in term 1.
I20260812 21:03:38.074028 16069 leader_election.cc:302] T 0438574006eb490c80df916b1abecca3 P cb92e98ad74b42af916785483c36e382 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92601044a2eb4651b9d3bb09cbe9d5b0, cb92e98ad74b42af916785483c36e382; no voters:
I20260812 21:03:38.074244 16425 raft_consensus.cc:2781] T 0438574006eb490c80df916b1abecca3 P cb92e98ad74b42af916785483c36e382 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:03:38.074326 16425 raft_consensus.cc:686] T 0438574006eb490c80df916b1abecca3 P cb92e98ad74b42af916785483c36e382 [term 1 LEADER]: Becoming Leader. State: Replica: cb92e98ad74b42af916785483c36e382, State: Running, Role: LEADER
I20260812 21:03:38.074374 16425 consensus_queue.cc:237] T 0438574006eb490c80df916b1abecca3 P cb92e98ad74b42af916785483c36e382 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92601044a2eb4651b9d3bb09cbe9d5b0" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 33109 } } peers { permanent_uuid: "ca2f0c3dd5b0431a913c99ba5bfd2bd5" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 44949 } } peers { permanent_uuid: "cb92e98ad74b42af916785483c36e382" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 35785 } }
I20260812 21:03:38.074992 15361 catalog_manager.cc:5495] T 0438574006eb490c80df916b1abecca3 P cb92e98ad74b42af916785483c36e382 reported cstate change: term changed from 0 to 1, leader changed from <none> to cb92e98ad74b42af916785483c36e382 (127.11.254.131). New cstate: current_term: 1 leader_uuid: "cb92e98ad74b42af916785483c36e382" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92601044a2eb4651b9d3bb09cbe9d5b0" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 33109 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ca2f0c3dd5b0431a913c99ba5bfd2bd5" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 44949 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb92e98ad74b42af916785483c36e382" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 35785 } health_report { overall_health: HEALTHY } } }
I20260812 21:03:38.082639 15944 raft_consensus.cc:1261] T 0438574006eb490c80df916b1abecca3 P ca2f0c3dd5b0431a913c99ba5bfd2bd5 [term 1 FOLLOWER]: Refusing update from remote peer cb92e98ad74b42af916785483c36e382: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 21:03:38.082639 15670 raft_consensus.cc:1261] T 0438574006eb490c80df916b1abecca3 P 92601044a2eb4651b9d3bb09cbe9d5b0 [term 1 FOLLOWER]: Refusing update from remote peer cb92e98ad74b42af916785483c36e382: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 21:03:38.082892 16420 consensus_queue.cc:1035] T 0438574006eb490c80df916b1abecca3 P cb92e98ad74b42af916785483c36e382 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca2f0c3dd5b0431a913c99ba5bfd2bd5" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 44949 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 21:03:38.083343 16420 consensus_queue.cc:1035] T 0438574006eb490c80df916b1abecca3 P cb92e98ad74b42af916785483c36e382 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92601044a2eb4651b9d3bb09cbe9d5b0" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 33109 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260812 21:03:38.088963 16391 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260812 21:03:39.612819 15357 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35844:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 21:03:39.612926 15357 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35844:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 21:03:39.613718 15357 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f41525e3e1134a4887fe0909de2f21f6: Sending DeleteTablet for 3 replicas of tablet 0438574006eb490c80df916b1abecca3
I20260812 21:03:39.614013 15630 tablet_service.cc:1514] Processing DeleteTablet for tablet 0438574006eb490c80df916b1abecca3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:03:39 UTC) from {username='kudu', principal='kudu/127.11.254.190@KRBTEST.COM'} at 127.0.0.1:40914
I20260812 21:03:39.614027 16169 tablet_service.cc:1514] Processing DeleteTablet for tablet 0438574006eb490c80df916b1abecca3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:03:39 UTC) from {username='kudu', principal='kudu/127.11.254.190@KRBTEST.COM'} at 127.0.0.1:57404
I20260812 21:03:39.614110 15906 tablet_service.cc:1514] Processing DeleteTablet for tablet 0438574006eb490c80df916b1abecca3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:03:39 UTC) from {username='kudu', principal='kudu/127.11.254.190@KRBTEST.COM'} at 127.0.0.1:53566
I20260812 21:03:39.614226 16492 tablet_replica.cc:330] T 0438574006eb490c80df916b1abecca3 P cb92e98ad74b42af916785483c36e382: stopping tablet replica
I20260812 21:03:39.614276 16492 raft_consensus.cc:2229] T 0438574006eb490c80df916b1abecca3 P cb92e98ad74b42af916785483c36e382 [term 1 LEADER]: Raft consensus shutting down.
I20260812 21:03:39.614331 16492 raft_consensus.cc:2258] T 0438574006eb490c80df916b1abecca3 P cb92e98ad74b42af916785483c36e382 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 21:03:39.614549 16492 ts_tablet_manager.cc:1905] T 0438574006eb490c80df916b1abecca3 P cb92e98ad74b42af916785483c36e382: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 21:03:39.616335 16492 ts_tablet_manager.cc:1918] T 0438574006eb490c80df916b1abecca3 P cb92e98ad74b42af916785483c36e382: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 21:03:39.616392 16492 log.cc:1196] T 0438574006eb490c80df916b1abecca3 P cb92e98ad74b42af916785483c36e382: Deleting WAL directory at /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568607354921-12282-0/minicluster-data/ts-2/wal/wals/0438574006eb490c80df916b1abecca3
I20260812 21:03:39.616779 16492 ts_tablet_manager.cc:1939] T 0438574006eb490c80df916b1abecca3 P cb92e98ad74b42af916785483c36e382: Deleting consensus metadata
I20260812 21:03:39.617139 15338 catalog_manager.cc:4841] TS cb92e98ad74b42af916785483c36e382 (127.11.254.131:35785): tablet 0438574006eb490c80df916b1abecca3 (table test-table [id=026bd7b41f4f4a509ae967322d644e74]) successfully deleted
I20260812 21:03:39.618403 16493 tablet_replica.cc:330] T 0438574006eb490c80df916b1abecca3 P 92601044a2eb4651b9d3bb09cbe9d5b0: stopping tablet replica
I20260812 21:03:39.618486 16493 raft_consensus.cc:2229] T 0438574006eb490c80df916b1abecca3 P 92601044a2eb4651b9d3bb09cbe9d5b0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 21:03:39.618520 16493 raft_consensus.cc:2258] T 0438574006eb490c80df916b1abecca3 P 92601044a2eb4651b9d3bb09cbe9d5b0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 21:03:39.618613 16495 tablet_replica.cc:330] T 0438574006eb490c80df916b1abecca3 P ca2f0c3dd5b0431a913c99ba5bfd2bd5: stopping tablet replica
I20260812 21:03:39.618675 16495 raft_consensus.cc:2229] T 0438574006eb490c80df916b1abecca3 P ca2f0c3dd5b0431a913c99ba5bfd2bd5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 21:03:39.618712 16495 raft_consensus.cc:2258] T 0438574006eb490c80df916b1abecca3 P ca2f0c3dd5b0431a913c99ba5bfd2bd5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 21:03:39.619009 16493 ts_tablet_manager.cc:1905] T 0438574006eb490c80df916b1abecca3 P 92601044a2eb4651b9d3bb09cbe9d5b0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 21:03:39.620759 16493 ts_tablet_manager.cc:1918] T 0438574006eb490c80df916b1abecca3 P 92601044a2eb4651b9d3bb09cbe9d5b0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 21:03:39.620813 16493 log.cc:1196] T 0438574006eb490c80df916b1abecca3 P 92601044a2eb4651b9d3bb09cbe9d5b0: Deleting WAL directory at /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568607354921-12282-0/minicluster-data/ts-0/wal/wals/0438574006eb490c80df916b1abecca3
I20260812 21:03:39.621003 16493 ts_tablet_manager.cc:1939] T 0438574006eb490c80df916b1abecca3 P 92601044a2eb4651b9d3bb09cbe9d5b0: Deleting consensus metadata
I20260812 21:03:39.621222 16495 ts_tablet_manager.cc:1905] T 0438574006eb490c80df916b1abecca3 P ca2f0c3dd5b0431a913c99ba5bfd2bd5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 21:03:39.621472 15334 catalog_manager.cc:4841] TS 92601044a2eb4651b9d3bb09cbe9d5b0 (127.11.254.129:33109): tablet 0438574006eb490c80df916b1abecca3 (table test-table [id=026bd7b41f4f4a509ae967322d644e74]) successfully deleted
I20260812 21:03:39.622591 16495 ts_tablet_manager.cc:1918] T 0438574006eb490c80df916b1abecca3 P ca2f0c3dd5b0431a913c99ba5bfd2bd5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 21:03:39.622643 16495 log.cc:1196] T 0438574006eb490c80df916b1abecca3 P ca2f0c3dd5b0431a913c99ba5bfd2bd5: Deleting WAL directory at /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568607354921-12282-0/minicluster-data/ts-1/wal/wals/0438574006eb490c80df916b1abecca3
I20260812 21:03:39.622781 16495 ts_tablet_manager.cc:1939] T 0438574006eb490c80df916b1abecca3 P ca2f0c3dd5b0431a913c99ba5bfd2bd5: Deleting consensus metadata
I20260812 21:03:39.623227 15338 catalog_manager.cc:4841] TS ca2f0c3dd5b0431a913c99ba5bfd2bd5 (127.11.254.130:44949): tablet 0438574006eb490c80df916b1abecca3 (table test-table [id=026bd7b41f4f4a509ae967322d644e74]) successfully deleted
Aug 12 21:03:39 dist-test-slave-vpvm krb5kdc[15282](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568619, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 21:03:39 dist-test-slave-vpvm krb5kdc[15282](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568619, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.11.254.190@KRBTEST.COM
I20260812 21:03:39.644784 16507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:39.632594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53836 (local address 127.11.254.190:45761)
0812 21:03:39.632736 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0812 21:03:39.632738 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:03:39.632747 (+ 9us) server_negotiation.cc:407] Connection header received
0812 21:03:39.632774 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:03:39.632774 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:03:39.632786 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:03:39.632846 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:03:39.633373 (+ 527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:39.633896 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:39.634643 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:39.634853 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:39.634906 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:39.642667 (+ 7761us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:03:39.642669 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:03:39.642669 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:03:39.642678 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:03:39.643890 (+ 1212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:39.644192 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:39.644192 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:39.644193 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:39.644240 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:39.644367 (+ 127us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:39.644367 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:39.644368 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:39.644475 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:03:39.644542 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 21:03:39.644633 (+ 91us) server_negotiation.cc:299] Negotiation successful
0812 21:03:39.644702 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":92,"threads_started":1}
I20260812 21:03:39.646332 15357 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53836:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 21:03:39.646488 15357 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 21:03:39.647830 15906 tablet_service.cc:1467] Processing CreateTablet for tablet 32f77328a81649f5bf68d198e45d62a2 (DEFAULT_TABLE table=test-table [id=c0615aa56cbf411d96b4f3161f8f7cdf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:03:39.647846 15630 tablet_service.cc:1467] Processing CreateTablet for tablet 32f77328a81649f5bf68d198e45d62a2 (DEFAULT_TABLE table=test-table [id=c0615aa56cbf411d96b4f3161f8f7cdf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:03:39.647967 15906 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 32f77328a81649f5bf68d198e45d62a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:03:39.647972 15630 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 32f77328a81649f5bf68d198e45d62a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:03:39.648974 16508 tablet_bootstrap.cc:492] T 32f77328a81649f5bf68d198e45d62a2 P ca2f0c3dd5b0431a913c99ba5bfd2bd5: Bootstrap starting.
I20260812 21:03:39.649039 16169 tablet_service.cc:1467] Processing CreateTablet for tablet 32f77328a81649f5bf68d198e45d62a2 (DEFAULT_TABLE table=test-table [id=c0615aa56cbf411d96b4f3161f8f7cdf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:03:39.649003 16510 tablet_bootstrap.cc:492] T 32f77328a81649f5bf68d198e45d62a2 P 92601044a2eb4651b9d3bb09cbe9d5b0: Bootstrap starting.
I20260812 21:03:39.649255 16169 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 32f77328a81649f5bf68d198e45d62a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:03:39.649466 16508 tablet_bootstrap.cc:654] T 32f77328a81649f5bf68d198e45d62a2 P ca2f0c3dd5b0431a913c99ba5bfd2bd5: Neither blocks nor log segments found. Creating new log.
I20260812 21:03:39.649466 16510 tablet_bootstrap.cc:654] T 32f77328a81649f5bf68d198e45d62a2 P 92601044a2eb4651b9d3bb09cbe9d5b0: Neither blocks nor log segments found. Creating new log.
I20260812 21:03:39.650085 16508 tablet_bootstrap.cc:492] T 32f77328a81649f5bf68d198e45d62a2 P ca2f0c3dd5b0431a913c99ba5bfd2bd5: No bootstrap required, opened a new log
I20260812 21:03:39.650141 16508 ts_tablet_manager.cc:1397] T 32f77328a81649f5bf68d198e45d62a2 P ca2f0c3dd5b0431a913c99ba5bfd2bd5: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 21:03:39.650270 16510 tablet_bootstrap.cc:492] T 32f77328a81649f5bf68d198e45d62a2 P 92601044a2eb4651b9d3bb09cbe9d5b0: No bootstrap required, opened a new log
I20260812 21:03:39.650322 16510 ts_tablet_manager.cc:1397] T 32f77328a81649f5bf68d198e45d62a2 P 92601044a2eb4651b9d3bb09cbe9d5b0: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 21:03:39.650283 16508 raft_consensus.cc:348] T 32f77328a81649f5bf68d198e45d62a2 P ca2f0c3dd5b0431a913c99ba5bfd2bd5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92601044a2eb4651b9d3bb09cbe9d5b0" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 33109 } } peers { permanent_uuid: "ca2f0c3dd5b0431a913c99ba5bfd2bd5" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 44949 } } peers { permanent_uuid: "cb92e98ad74b42af916785483c36e382" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 35785 } }
I20260812 21:03:39.650355 16508 raft_consensus.cc:374] T 32f77328a81649f5bf68d198e45d62a2 P ca2f0c3dd5b0431a913c99ba5bfd2bd5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:03:39.650383 16508 raft_consensus.cc:729] T 32f77328a81649f5bf68d198e45d62a2 P ca2f0c3dd5b0431a913c99ba5bfd2bd5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca2f0c3dd5b0431a913c99ba5bfd2bd5, State: Initialized, Role: FOLLOWER
I20260812 21:03:39.650421 16508 consensus_queue.cc:260] T 32f77328a81649f5bf68d198e45d62a2 P ca2f0c3dd5b0431a913c99ba5bfd2bd5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92601044a2eb4651b9d3bb09cbe9d5b0" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 33109 } } peers { permanent_uuid: "ca2f0c3dd5b0431a913c99ba5bfd2bd5" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 44949 } } peers { permanent_uuid: "cb92e98ad74b42af916785483c36e382" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 35785 } }
I20260812 21:03:39.650457 16510 raft_consensus.cc:348] T 32f77328a81649f5bf68d198e45d62a2 P 92601044a2eb4651b9d3bb09cbe9d5b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92601044a2eb4651b9d3bb09cbe9d5b0" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 33109 } } peers { permanent_uuid: "ca2f0c3dd5b0431a913c99ba5bfd2bd5" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 44949 } } peers { permanent_uuid: "cb92e98ad74b42af916785483c36e382" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 35785 } }
I20260812 21:03:39.650528 16510 raft_consensus.cc:374] T 32f77328a81649f5bf68d198e45d62a2 P 92601044a2eb4651b9d3bb09cbe9d5b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:03:39.650553 16510 raft_consensus.cc:729] T 32f77328a81649f5bf68d198e45d62a2 P 92601044a2eb4651b9d3bb09cbe9d5b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92601044a2eb4651b9d3bb09cbe9d5b0, State: Initialized, Role: FOLLOWER
I20260812 21:03:39.650584 16508 ts_tablet_manager.cc:1428] T 32f77328a81649f5bf68d198e45d62a2 P ca2f0c3dd5b0431a913c99ba5bfd2bd5: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260812 21:03:39.650619 16510 consensus_queue.cc:260] T 32f77328a81649f5bf68d198e45d62a2 P 92601044a2eb4651b9d3bb09cbe9d5b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92601044a2eb4651b9d3bb09cbe9d5b0" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 33109 } } peers { permanent_uuid: "ca2f0c3dd5b0431a913c99ba5bfd2bd5" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 44949 } } peers { permanent_uuid: "cb92e98ad74b42af916785483c36e382" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 35785 } }
I20260812 21:03:39.650755 16510 ts_tablet_manager.cc:1428] T 32f77328a81649f5bf68d198e45d62a2 P 92601044a2eb4651b9d3bb09cbe9d5b0: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260812 21:03:39.651094 16520 tablet_bootstrap.cc:492] T 32f77328a81649f5bf68d198e45d62a2 P cb92e98ad74b42af916785483c36e382: Bootstrap starting.
I20260812 21:03:39.651712 16520 tablet_bootstrap.cc:654] T 32f77328a81649f5bf68d198e45d62a2 P cb92e98ad74b42af916785483c36e382: Neither blocks nor log segments found. Creating new log.
I20260812 21:03:39.652266 16520 tablet_bootstrap.cc:492] T 32f77328a81649f5bf68d198e45d62a2 P cb92e98ad74b42af916785483c36e382: No bootstrap required, opened a new log
I20260812 21:03:39.652311 16520 ts_tablet_manager.cc:1397] T 32f77328a81649f5bf68d198e45d62a2 P cb92e98ad74b42af916785483c36e382: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 21:03:39.652501 16520 raft_consensus.cc:348] T 32f77328a81649f5bf68d198e45d62a2 P cb92e98ad74b42af916785483c36e382 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92601044a2eb4651b9d3bb09cbe9d5b0" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 33109 } } peers { permanent_uuid: "ca2f0c3dd5b0431a913c99ba5bfd2bd5" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 44949 } } peers { permanent_uuid: "cb92e98ad74b42af916785483c36e382" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 35785 } }
I20260812 21:03:39.652573 16520 raft_consensus.cc:374] T 32f77328a81649f5bf68d198e45d62a2 P cb92e98ad74b42af916785483c36e382 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:03:39.652593 16520 raft_consensus.cc:729] T 32f77328a81649f5bf68d198e45d62a2 P cb92e98ad74b42af916785483c36e382 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb92e98ad74b42af916785483c36e382, State: Initialized, Role: FOLLOWER
I20260812 21:03:39.652635 16520 consensus_queue.cc:260] T 32f77328a81649f5bf68d198e45d62a2 P cb92e98ad74b42af916785483c36e382 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92601044a2eb4651b9d3bb09cbe9d5b0" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 33109 } } peers { permanent_uuid: "ca2f0c3dd5b0431a913c99ba5bfd2bd5" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 44949 } } peers { permanent_uuid: "cb92e98ad74b42af916785483c36e382" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 35785 } }
I20260812 21:03:39.652733 16520 ts_tablet_manager.cc:1428] T 32f77328a81649f5bf68d198e45d62a2 P cb92e98ad74b42af916785483c36e382: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 21:03:39.775305 16518 raft_consensus.cc:482] T 32f77328a81649f5bf68d198e45d62a2 P ca2f0c3dd5b0431a913c99ba5bfd2bd5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 21:03:39.775424 16518 raft_consensus.cc:504] T 32f77328a81649f5bf68d198e45d62a2 P ca2f0c3dd5b0431a913c99ba5bfd2bd5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92601044a2eb4651b9d3bb09cbe9d5b0" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 33109 } } peers { permanent_uuid: "ca2f0c3dd5b0431a913c99ba5bfd2bd5" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 44949 } } peers { permanent_uuid: "cb92e98ad74b42af916785483c36e382" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 35785 } }
I20260812 21:03:39.775787 16518 leader_election.cc:288] T 32f77328a81649f5bf68d198e45d62a2 P ca2f0c3dd5b0431a913c99ba5bfd2bd5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 92601044a2eb4651b9d3bb09cbe9d5b0 (127.11.254.129:33109), cb92e98ad74b42af916785483c36e382 (127.11.254.131:35785)
I20260812 21:03:39.778837 16524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:39.776032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.130:44773 (local address 127.11.254.131:35785)
0812 21:03:39.776215 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0812 21:03:39.776217 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:03:39.776291 (+ 74us) server_negotiation.cc:407] Connection header received
0812 21:03:39.776349 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:03:39.776350 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:03:39.776362 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:03:39.776427 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:03:39.776577 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:39.777128 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:39.778131 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:39.778505 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:39.778569 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:39.778578 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 21:03:39.778725 (+ 147us) server_negotiation.cc:299] Negotiation successful
0812 21:03:39.778766 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":114,"threads_started":1}
I20260812 21:03:39.778842 16527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:39.775970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.131:35785 (local address 127.11.254.130:44773)
0812 21:03:39.776214 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0812 21:03:39.776224 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 21:03:39.776291 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:03:39.776441 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:03:39.776442 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:03:39.776444 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:03:39.776515 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:39.776516 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:39.777210 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:39.777211 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:39.778042 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:39.778045 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:39.778580 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:39.778581 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:39.778663 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:39.778667 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 21:03:39.778713 (+ 46us) client_negotiation.cc:240] Negotiation successful
0812 21:03:39.778753 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":226,"mutex_wait_us":2,"thread_start_us":172,"threads_started":1}
I20260812 21:03:39.779203 16523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:39.775970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.129:33109 (local address 127.11.254.130:47633)
0812 21:03:39.776211 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0812 21:03:39.776223 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 21:03:39.776292 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:03:39.776435 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:03:39.776436 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:03:39.776438 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:03:39.776533 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:39.776534 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:39.777234 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:39.777235 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:39.778281 (+ 1046us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:39.778284 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:39.778938 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:39.778939 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:39.779019 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:39.779023 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 21:03:39.779080 (+ 57us) client_negotiation.cc:240] Negotiation successful
0812 21:03:39.779118 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":221,"mutex_wait_us":3,"thread_start_us":175,"threads_started":1}
I20260812 21:03:39.779186 16205 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32f77328a81649f5bf68d198e45d62a2" candidate_uuid: "ca2f0c3dd5b0431a913c99ba5bfd2bd5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb92e98ad74b42af916785483c36e382" is_pre_election: true
I20260812 21:03:39.779271 16205 raft_consensus.cc:2454] T 32f77328a81649f5bf68d198e45d62a2 P cb92e98ad74b42af916785483c36e382 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca2f0c3dd5b0431a913c99ba5bfd2bd5 in term 0.
I20260812 21:03:39.779276 16529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:39.776078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.130:47633 (local address 127.11.254.129:33109)
0812 21:03:39.776215 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0812 21:03:39.776217 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:03:39.776290 (+ 73us) server_negotiation.cc:407] Connection header received
0812 21:03:39.776349 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:03:39.776350 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:03:39.776361 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:03:39.776422 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:03:39.776595 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:39.777156 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:39.778373 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:39.778834 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:39.778903 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:39.778913 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0812 21:03:39.779157 (+ 244us) server_negotiation.cc:299] Negotiation successful
0812 21:03:39.779210 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":74,"threads_started":1}
I20260812 21:03:39.779448 15670 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32f77328a81649f5bf68d198e45d62a2" candidate_uuid: "ca2f0c3dd5b0431a913c99ba5bfd2bd5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92601044a2eb4651b9d3bb09cbe9d5b0" is_pre_election: true
I20260812 21:03:39.779549 15670 raft_consensus.cc:2454] T 32f77328a81649f5bf68d198e45d62a2 P 92601044a2eb4651b9d3bb09cbe9d5b0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca2f0c3dd5b0431a913c99ba5bfd2bd5 in term 0.
I20260812 21:03:39.779505 15811 leader_election.cc:302] T 32f77328a81649f5bf68d198e45d62a2 P ca2f0c3dd5b0431a913c99ba5bfd2bd5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ca2f0c3dd5b0431a913c99ba5bfd2bd5, cb92e98ad74b42af916785483c36e382; no voters:
I20260812 21:03:39.779651 16518 raft_consensus.cc:2781] T 32f77328a81649f5bf68d198e45d62a2 P ca2f0c3dd5b0431a913c99ba5bfd2bd5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 21:03:39.779697 16518 raft_consensus.cc:482] T 32f77328a81649f5bf68d198e45d62a2 P ca2f0c3dd5b0431a913c99ba5bfd2bd5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 21:03:39.779729 16518 raft_consensus.cc:3037] T 32f77328a81649f5bf68d198e45d62a2 P ca2f0c3dd5b0431a913c99ba5bfd2bd5 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:03:39.780537 16518 raft_consensus.cc:504] T 32f77328a81649f5bf68d198e45d62a2 P ca2f0c3dd5b0431a913c99ba5bfd2bd5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92601044a2eb4651b9d3bb09cbe9d5b0" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 33109 } } peers { permanent_uuid: "ca2f0c3dd5b0431a913c99ba5bfd2bd5" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 44949 } } peers { permanent_uuid: "cb92e98ad74b42af916785483c36e382" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 35785 } }
I20260812 21:03:39.780666 16518 leader_election.cc:288] T 32f77328a81649f5bf68d198e45d62a2 P ca2f0c3dd5b0431a913c99ba5bfd2bd5 [CANDIDATE]: Term 1 election: Requested vote from peers 92601044a2eb4651b9d3bb09cbe9d5b0 (127.11.254.129:33109), cb92e98ad74b42af916785483c36e382 (127.11.254.131:35785)
I20260812 21:03:39.780814 16205 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32f77328a81649f5bf68d198e45d62a2" candidate_uuid: "ca2f0c3dd5b0431a913c99ba5bfd2bd5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb92e98ad74b42af916785483c36e382"
I20260812 21:03:39.780819 15670 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32f77328a81649f5bf68d198e45d62a2" candidate_uuid: "ca2f0c3dd5b0431a913c99ba5bfd2bd5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92601044a2eb4651b9d3bb09cbe9d5b0"
I20260812 21:03:39.780884 16205 raft_consensus.cc:3037] T 32f77328a81649f5bf68d198e45d62a2 P cb92e98ad74b42af916785483c36e382 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:03:39.780893 15670 raft_consensus.cc:3037] T 32f77328a81649f5bf68d198e45d62a2 P 92601044a2eb4651b9d3bb09cbe9d5b0 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:03:39.781450 16205 raft_consensus.cc:2454] T 32f77328a81649f5bf68d198e45d62a2 P cb92e98ad74b42af916785483c36e382 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca2f0c3dd5b0431a913c99ba5bfd2bd5 in term 1.
I20260812 21:03:39.781519 15670 raft_consensus.cc:2454] T 32f77328a81649f5bf68d198e45d62a2 P 92601044a2eb4651b9d3bb09cbe9d5b0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca2f0c3dd5b0431a913c99ba5bfd2bd5 in term 1.
I20260812 21:03:39.781644 15811 leader_election.cc:302] T 32f77328a81649f5bf68d198e45d62a2 P ca2f0c3dd5b0431a913c99ba5bfd2bd5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ca2f0c3dd5b0431a913c99ba5bfd2bd5, cb92e98ad74b42af916785483c36e382; no voters:
I20260812 21:03:39.781744 16518 raft_consensus.cc:2781] T 32f77328a81649f5bf68d198e45d62a2 P ca2f0c3dd5b0431a913c99ba5bfd2bd5 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:03:39.781972 16518 raft_consensus.cc:686] T 32f77328a81649f5bf68d198e45d62a2 P ca2f0c3dd5b0431a913c99ba5bfd2bd5 [term 1 LEADER]: Becoming Leader. State: Replica: ca2f0c3dd5b0431a913c99ba5bfd2bd5, State: Running, Role: LEADER
I20260812 21:03:39.782063 16518 consensus_queue.cc:237] T 32f77328a81649f5bf68d198e45d62a2 P ca2f0c3dd5b0431a913c99ba5bfd2bd5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92601044a2eb4651b9d3bb09cbe9d5b0" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 33109 } } peers { permanent_uuid: "ca2f0c3dd5b0431a913c99ba5bfd2bd5" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 44949 } } peers { permanent_uuid: "cb92e98ad74b42af916785483c36e382" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 35785 } }
I20260812 21:03:39.782814 15361 catalog_manager.cc:5495] T 32f77328a81649f5bf68d198e45d62a2 P ca2f0c3dd5b0431a913c99ba5bfd2bd5 reported cstate change: term changed from 0 to 1, leader changed from <none> to ca2f0c3dd5b0431a913c99ba5bfd2bd5 (127.11.254.130). New cstate: current_term: 1 leader_uuid: "ca2f0c3dd5b0431a913c99ba5bfd2bd5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92601044a2eb4651b9d3bb09cbe9d5b0" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 33109 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ca2f0c3dd5b0431a913c99ba5bfd2bd5" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 44949 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb92e98ad74b42af916785483c36e382" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 35785 } health_report { overall_health: UNKNOWN } } }
I20260812 21:03:39.799480 16535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:39.796743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37310 (local address 127.11.254.130:44949)
0812 21:03:39.797058 (+ 315us) server_negotiation.cc:206] Beginning negotiation
0812 21:03:39.797060 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:03:39.797068 (+ 8us) server_negotiation.cc:407] Connection header received
0812 21:03:39.797123 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:03:39.797124 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:03:39.797145 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:03:39.797223 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 21:03:39.797382 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:39.797976 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:39.798756 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:39.798959 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:39.799014 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:39.799137 (+ 123us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 21:03:39.799248 (+ 111us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 21:03:39.799302 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0812 21:03:39.799344 (+ 42us) server_negotiation.cc:299] Negotiation successful
0812 21:03:39.799363 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":292,"thread_start_us":236,"threads_started":1}
I20260812 21:03:39.804805 16529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:39.802452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.130:54047 (local address 127.11.254.129:33109)
0812 21:03:39.802505 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0812 21:03:39.802507 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:03:39.802625 (+ 118us) server_negotiation.cc:407] Connection header received
0812 21:03:39.802657 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:03:39.802658 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:03:39.802669 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:03:39.802725 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 21:03:39.802823 (+ 98us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:39.803298 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:39.804119 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:39.804320 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:39.804375 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:39.804537 (+ 162us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 21:03:39.804577 (+ 40us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 21:03:39.804628 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0812 21:03:39.804718 (+ 90us) server_negotiation.cc:299] Negotiation successful
0812 21:03:39.804742 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260812 21:03:39.804822 16542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:39.802356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.129:33109 (local address 127.11.254.130:54047)
0812 21:03:39.802525 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0812 21:03:39.802535 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 21:03:39.802588 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:03:39.802731 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:03:39.802732 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:03:39.802733 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 21:03:39.802785 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:39.802786 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:39.803345 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:39.803346 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:39.804027 (+ 681us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:39.804030 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:39.804381 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:39.804381 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:39.804428 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:39.804442 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 21:03:39.804671 (+ 229us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 21:03:39.804674 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 21:03:39.804710 (+ 36us) client_negotiation.cc:240] Negotiation successful
0812 21:03:39.804754 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":73,"threads_started":1}
I20260812 21:03:39.807143 15670 raft_consensus.cc:1261] T 32f77328a81649f5bf68d198e45d62a2 P 92601044a2eb4651b9d3bb09cbe9d5b0 [term 1 FOLLOWER]: Refusing update from remote peer ca2f0c3dd5b0431a913c99ba5bfd2bd5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 21:03:39.807147 16205 raft_consensus.cc:1261] T 32f77328a81649f5bf68d198e45d62a2 P cb92e98ad74b42af916785483c36e382 [term 1 FOLLOWER]: Refusing update from remote peer ca2f0c3dd5b0431a913c99ba5bfd2bd5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 21:03:39.807405 16532 consensus_queue.cc:1035] T 32f77328a81649f5bf68d198e45d62a2 P ca2f0c3dd5b0431a913c99ba5bfd2bd5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb92e98ad74b42af916785483c36e382" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 35785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 21:03:39.807536 16518 consensus_queue.cc:1035] T 32f77328a81649f5bf68d198e45d62a2 P ca2f0c3dd5b0431a913c99ba5bfd2bd5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92601044a2eb4651b9d3bb09cbe9d5b0" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 33109 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 21:03:39.815164 16535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:39.812774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.129:54387 (local address 127.11.254.130:44949)
0812 21:03:39.812813 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0812 21:03:39.812815 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:03:39.812878 (+ 63us) server_negotiation.cc:407] Connection header received
0812 21:03:39.812943 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:03:39.812944 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:03:39.812956 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:03:39.813007 (+ 51us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 21:03:39.813114 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:39.813591 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:39.814411 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:39.814657 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:39.814713 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:39.814926 (+ 213us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 21:03:39.814966 (+ 40us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 21:03:39.815017 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0812 21:03:39.815085 (+ 68us) server_negotiation.cc:299] Negotiation successful
0812 21:03:39.815106 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260812 21:03:39.815187 16554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:39.812644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.130:44949 (local address 127.11.254.129:54387)
0812 21:03:39.812823 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0812 21:03:39.812834 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 21:03:39.812893 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:03:39.813027 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:03:39.813028 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:03:39.813030 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 21:03:39.813076 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:39.813078 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:39.813651 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:39.813651 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:39.814338 (+ 687us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:39.814340 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:39.814783 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:39.814783 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:39.814831 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:39.814835 (+ 4us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 21:03:39.815026 (+ 191us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 21:03:39.815028 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 21:03:39.815081 (+ 53us) client_negotiation.cc:240] Negotiation successful
0812 21:03:39.815125 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":80,"threads_started":1}
I20260812 21:03:39.821780 15355 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53836:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 21:03:39.821907 15355 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53836:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 21:03:39.822641 15355 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f41525e3e1134a4887fe0909de2f21f6: Sending DeleteTablet for 3 replicas of tablet 32f77328a81649f5bf68d198e45d62a2
I20260812 21:03:39.822923 15630 tablet_service.cc:1514] Processing DeleteTablet for tablet 32f77328a81649f5bf68d198e45d62a2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:03:39 UTC) from {username='kudu', principal='kudu/127.11.254.190@KRBTEST.COM'} at 127.0.0.1:40914
I20260812 21:03:39.822918 16169 tablet_service.cc:1514] Processing DeleteTablet for tablet 32f77328a81649f5bf68d198e45d62a2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:03:39 UTC) from {username='kudu', principal='kudu/127.11.254.190@KRBTEST.COM'} at 127.0.0.1:57404
I20260812 21:03:39.823032 16493 tablet_replica.cc:330] T 32f77328a81649f5bf68d198e45d62a2 P 92601044a2eb4651b9d3bb09cbe9d5b0: stopping tablet replica
I20260812 21:03:39.823062 16492 tablet_replica.cc:330] T 32f77328a81649f5bf68d198e45d62a2 P cb92e98ad74b42af916785483c36e382: stopping tablet replica
I20260812 21:03:39.823086 15906 tablet_service.cc:1514] Processing DeleteTablet for tablet 32f77328a81649f5bf68d198e45d62a2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:03:39 UTC) from {username='kudu', principal='kudu/127.11.254.190@KRBTEST.COM'} at 127.0.0.1:53566
I20260812 21:03:39.823100 16493 raft_consensus.cc:2229] T 32f77328a81649f5bf68d198e45d62a2 P 92601044a2eb4651b9d3bb09cbe9d5b0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 21:03:39.823117 16492 raft_consensus.cc:2229] T 32f77328a81649f5bf68d198e45d62a2 P cb92e98ad74b42af916785483c36e382 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 21:03:39.823159 16493 raft_consensus.cc:2258] T 32f77328a81649f5bf68d198e45d62a2 P 92601044a2eb4651b9d3bb09cbe9d5b0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 21:03:39.823165 16492 raft_consensus.cc:2258] T 32f77328a81649f5bf68d198e45d62a2 P cb92e98ad74b42af916785483c36e382 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 21:03:39.823186 16495 tablet_replica.cc:330] T 32f77328a81649f5bf68d198e45d62a2 P ca2f0c3dd5b0431a913c99ba5bfd2bd5: stopping tablet replica
I20260812 21:03:39.823242 16495 raft_consensus.cc:2229] T 32f77328a81649f5bf68d198e45d62a2 P ca2f0c3dd5b0431a913c99ba5bfd2bd5 [term 1 LEADER]: Raft consensus shutting down.
I20260812 21:03:39.823345 16495 raft_consensus.cc:2258] T 32f77328a81649f5bf68d198e45d62a2 P ca2f0c3dd5b0431a913c99ba5bfd2bd5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 21:03:39.823611 16492 ts_tablet_manager.cc:1905] T 32f77328a81649f5bf68d198e45d62a2 P cb92e98ad74b42af916785483c36e382: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 21:03:39.823740 16493 ts_tablet_manager.cc:1905] T 32f77328a81649f5bf68d198e45d62a2 P 92601044a2eb4651b9d3bb09cbe9d5b0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 21:03:39.824988 16495 ts_tablet_manager.cc:1905] T 32f77328a81649f5bf68d198e45d62a2 P ca2f0c3dd5b0431a913c99ba5bfd2bd5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 21:03:39.825012 16492 ts_tablet_manager.cc:1918] T 32f77328a81649f5bf68d198e45d62a2 P cb92e98ad74b42af916785483c36e382: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 21:03:39.825073 16492 log.cc:1196] T 32f77328a81649f5bf68d198e45d62a2 P cb92e98ad74b42af916785483c36e382: Deleting WAL directory at /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568607354921-12282-0/minicluster-data/ts-2/wal/wals/32f77328a81649f5bf68d198e45d62a2
I20260812 21:03:39.825279 16492 ts_tablet_manager.cc:1939] T 32f77328a81649f5bf68d198e45d62a2 P cb92e98ad74b42af916785483c36e382: Deleting consensus metadata
I20260812 21:03:39.825451 16493 ts_tablet_manager.cc:1918] T 32f77328a81649f5bf68d198e45d62a2 P 92601044a2eb4651b9d3bb09cbe9d5b0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 21:03:39.825503 16493 log.cc:1196] T 32f77328a81649f5bf68d198e45d62a2 P 92601044a2eb4651b9d3bb09cbe9d5b0: Deleting WAL directory at /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568607354921-12282-0/minicluster-data/ts-0/wal/wals/32f77328a81649f5bf68d198e45d62a2
I20260812 21:03:39.825718 16493 ts_tablet_manager.cc:1939] T 32f77328a81649f5bf68d198e45d62a2 P 92601044a2eb4651b9d3bb09cbe9d5b0: Deleting consensus metadata
I20260812 21:03:39.826104 15338 catalog_manager.cc:4841] TS cb92e98ad74b42af916785483c36e382 (127.11.254.131:35785): tablet 32f77328a81649f5bf68d198e45d62a2 (table test-table [id=c0615aa56cbf411d96b4f3161f8f7cdf]) successfully deleted
I20260812 21:03:39.826152 15334 catalog_manager.cc:4841] TS 92601044a2eb4651b9d3bb09cbe9d5b0 (127.11.254.129:33109): tablet 32f77328a81649f5bf68d198e45d62a2 (table test-table [id=c0615aa56cbf411d96b4f3161f8f7cdf]) successfully deleted
I20260812 21:03:39.826319 15355 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53836:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 21:03:39.826478 15355 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 21:03:39.827000 16495 ts_tablet_manager.cc:1918] T 32f77328a81649f5bf68d198e45d62a2 P ca2f0c3dd5b0431a913c99ba5bfd2bd5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 21:03:39.827056 16495 log.cc:1196] T 32f77328a81649f5bf68d198e45d62a2 P ca2f0c3dd5b0431a913c99ba5bfd2bd5: Deleting WAL directory at /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568607354921-12282-0/minicluster-data/ts-1/wal/wals/32f77328a81649f5bf68d198e45d62a2
I20260812 21:03:39.827263 16495 ts_tablet_manager.cc:1939] T 32f77328a81649f5bf68d198e45d62a2 P ca2f0c3dd5b0431a913c99ba5bfd2bd5: Deleting consensus metadata
I20260812 21:03:39.827554 15338 catalog_manager.cc:4841] TS ca2f0c3dd5b0431a913c99ba5bfd2bd5 (127.11.254.130:44949): tablet 32f77328a81649f5bf68d198e45d62a2 (table test-table [id=c0615aa56cbf411d96b4f3161f8f7cdf]) successfully deleted
I20260812 21:03:39.828229 15630 tablet_service.cc:1467] Processing CreateTablet for tablet f3466ece79414d0ebd1144531d0ddd25 (DEFAULT_TABLE table=test-table [id=96e01394bb8c4585a399e855e99ffe5a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:03:39.828244 16169 tablet_service.cc:1467] Processing CreateTablet for tablet f3466ece79414d0ebd1144531d0ddd25 (DEFAULT_TABLE table=test-table [id=96e01394bb8c4585a399e855e99ffe5a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:03:39.828261 15906 tablet_service.cc:1467] Processing CreateTablet for tablet f3466ece79414d0ebd1144531d0ddd25 (DEFAULT_TABLE table=test-table [id=96e01394bb8c4585a399e855e99ffe5a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:03:39.828347 16169 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f3466ece79414d0ebd1144531d0ddd25. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:03:39.828346 15630 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f3466ece79414d0ebd1144531d0ddd25. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:03:39.828357 15906 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f3466ece79414d0ebd1144531d0ddd25. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:03:39.829470 16508 tablet_bootstrap.cc:492] T f3466ece79414d0ebd1144531d0ddd25 P ca2f0c3dd5b0431a913c99ba5bfd2bd5: Bootstrap starting.
I20260812 21:03:39.829493 16520 tablet_bootstrap.cc:492] T f3466ece79414d0ebd1144531d0ddd25 P cb92e98ad74b42af916785483c36e382: Bootstrap starting.
I20260812 21:03:39.829795 16520 tablet_bootstrap.cc:654] T f3466ece79414d0ebd1144531d0ddd25 P cb92e98ad74b42af916785483c36e382: Neither blocks nor log segments found. Creating new log.
I20260812 21:03:39.830416 16508 tablet_bootstrap.cc:654] T f3466ece79414d0ebd1144531d0ddd25 P ca2f0c3dd5b0431a913c99ba5bfd2bd5: Neither blocks nor log segments found. Creating new log.
I20260812 21:03:39.831015 16510 tablet_bootstrap.cc:492] T f3466ece79414d0ebd1144531d0ddd25 P 92601044a2eb4651b9d3bb09cbe9d5b0: Bootstrap starting.
I20260812 21:03:39.831336 16508 tablet_bootstrap.cc:492] T f3466ece79414d0ebd1144531d0ddd25 P ca2f0c3dd5b0431a913c99ba5bfd2bd5: No bootstrap required, opened a new log
I20260812 21:03:39.831385 16508 ts_tablet_manager.cc:1397] T f3466ece79414d0ebd1144531d0ddd25 P ca2f0c3dd5b0431a913c99ba5bfd2bd5: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260812 21:03:39.831432 16510 tablet_bootstrap.cc:654] T f3466ece79414d0ebd1144531d0ddd25 P 92601044a2eb4651b9d3bb09cbe9d5b0: Neither blocks nor log segments found. Creating new log.
I20260812 21:03:39.831972 16510 tablet_bootstrap.cc:492] T f3466ece79414d0ebd1144531d0ddd25 P 92601044a2eb4651b9d3bb09cbe9d5b0: No bootstrap required, opened a new log
I20260812 21:03:39.831938 16508 raft_consensus.cc:348] T f3466ece79414d0ebd1144531d0ddd25 P ca2f0c3dd5b0431a913c99ba5bfd2bd5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92601044a2eb4651b9d3bb09cbe9d5b0" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 33109 } } peers { permanent_uuid: "ca2f0c3dd5b0431a913c99ba5bfd2bd5" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 44949 } } peers { permanent_uuid: "cb92e98ad74b42af916785483c36e382" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 35785 } }
I20260812 21:03:39.832016 16510 ts_tablet_manager.cc:1397] T f3466ece79414d0ebd1144531d0ddd25 P 92601044a2eb4651b9d3bb09cbe9d5b0: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 21:03:39.832020 16508 raft_consensus.cc:374] T f3466ece79414d0ebd1144531d0ddd25 P ca2f0c3dd5b0431a913c99ba5bfd2bd5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:03:39.832046 16508 raft_consensus.cc:729] T f3466ece79414d0ebd1144531d0ddd25 P ca2f0c3dd5b0431a913c99ba5bfd2bd5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca2f0c3dd5b0431a913c99ba5bfd2bd5, State: Initialized, Role: FOLLOWER
I20260812 21:03:39.832103 16508 consensus_queue.cc:260] T f3466ece79414d0ebd1144531d0ddd25 P ca2f0c3dd5b0431a913c99ba5bfd2bd5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92601044a2eb4651b9d3bb09cbe9d5b0" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 33109 } } peers { permanent_uuid: "ca2f0c3dd5b0431a913c99ba5bfd2bd5" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 44949 } } peers { permanent_uuid: "cb92e98ad74b42af916785483c36e382" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 35785 } }
I20260812 21:03:39.832154 16520 tablet_bootstrap.cc:492] T f3466ece79414d0ebd1144531d0ddd25 P cb92e98ad74b42af916785483c36e382: No bootstrap required, opened a new log
I20260812 21:03:39.832118 16510 raft_consensus.cc:348] T f3466ece79414d0ebd1144531d0ddd25 P 92601044a2eb4651b9d3bb09cbe9d5b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92601044a2eb4651b9d3bb09cbe9d5b0" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 33109 } } peers { permanent_uuid: "ca2f0c3dd5b0431a913c99ba5bfd2bd5" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 44949 } } peers { permanent_uuid: "cb92e98ad74b42af916785483c36e382" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 35785 } }
I20260812 21:03:39.832183 16510 raft_consensus.cc:374] T f3466ece79414d0ebd1144531d0ddd25 P 92601044a2eb4651b9d3bb09cbe9d5b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:03:39.832193 16520 ts_tablet_manager.cc:1397] T f3466ece79414d0ebd1144531d0ddd25 P cb92e98ad74b42af916785483c36e382: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260812 21:03:39.832202 16510 raft_consensus.cc:729] T f3466ece79414d0ebd1144531d0ddd25 P 92601044a2eb4651b9d3bb09cbe9d5b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92601044a2eb4651b9d3bb09cbe9d5b0, State: Initialized, Role: FOLLOWER
I20260812 21:03:39.832248 16510 consensus_queue.cc:260] T f3466ece79414d0ebd1144531d0ddd25 P 92601044a2eb4651b9d3bb09cbe9d5b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92601044a2eb4651b9d3bb09cbe9d5b0" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 33109 } } peers { permanent_uuid: "ca2f0c3dd5b0431a913c99ba5bfd2bd5" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 44949 } } peers { permanent_uuid: "cb92e98ad74b42af916785483c36e382" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 35785 } }
I20260812 21:03:39.832329 16510 ts_tablet_manager.cc:1428] T f3466ece79414d0ebd1144531d0ddd25 P 92601044a2eb4651b9d3bb09cbe9d5b0: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 21:03:39.832327 16508 ts_tablet_manager.cc:1428] T f3466ece79414d0ebd1144531d0ddd25 P ca2f0c3dd5b0431a913c99ba5bfd2bd5: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260812 21:03:39.832304 16520 raft_consensus.cc:348] T f3466ece79414d0ebd1144531d0ddd25 P cb92e98ad74b42af916785483c36e382 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92601044a2eb4651b9d3bb09cbe9d5b0" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 33109 } } peers { permanent_uuid: "ca2f0c3dd5b0431a913c99ba5bfd2bd5" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 44949 } } peers { permanent_uuid: "cb92e98ad74b42af916785483c36e382" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 35785 } }
I20260812 21:03:39.832373 16520 raft_consensus.cc:374] T f3466ece79414d0ebd1144531d0ddd25 P cb92e98ad74b42af916785483c36e382 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:03:39.832398 16520 raft_consensus.cc:729] T f3466ece79414d0ebd1144531d0ddd25 P cb92e98ad74b42af916785483c36e382 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb92e98ad74b42af916785483c36e382, State: Initialized, Role: FOLLOWER
I20260812 21:03:39.832438 16520 consensus_queue.cc:260] T f3466ece79414d0ebd1144531d0ddd25 P cb92e98ad74b42af916785483c36e382 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92601044a2eb4651b9d3bb09cbe9d5b0" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 33109 } } peers { permanent_uuid: "ca2f0c3dd5b0431a913c99ba5bfd2bd5" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 44949 } } peers { permanent_uuid: "cb92e98ad74b42af916785483c36e382" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 35785 } }
I20260812 21:03:39.832520 16520 ts_tablet_manager.cc:1428] T f3466ece79414d0ebd1144531d0ddd25 P cb92e98ad74b42af916785483c36e382: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 21:03:39.851019 16532 raft_consensus.cc:482] T f3466ece79414d0ebd1144531d0ddd25 P ca2f0c3dd5b0431a913c99ba5bfd2bd5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 21:03:39.851131 16532 raft_consensus.cc:504] T f3466ece79414d0ebd1144531d0ddd25 P ca2f0c3dd5b0431a913c99ba5bfd2bd5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92601044a2eb4651b9d3bb09cbe9d5b0" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 33109 } } peers { permanent_uuid: "ca2f0c3dd5b0431a913c99ba5bfd2bd5" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 44949 } } peers { permanent_uuid: "cb92e98ad74b42af916785483c36e382" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 35785 } }
I20260812 21:03:39.851302 16532 leader_election.cc:288] T f3466ece79414d0ebd1144531d0ddd25 P ca2f0c3dd5b0431a913c99ba5bfd2bd5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 92601044a2eb4651b9d3bb09cbe9d5b0 (127.11.254.129:33109), cb92e98ad74b42af916785483c36e382 (127.11.254.131:35785)
I20260812 21:03:39.851512 15670 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f3466ece79414d0ebd1144531d0ddd25" candidate_uuid: "ca2f0c3dd5b0431a913c99ba5bfd2bd5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92601044a2eb4651b9d3bb09cbe9d5b0" is_pre_election: true
I20260812 21:03:39.851518 16205 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f3466ece79414d0ebd1144531d0ddd25" candidate_uuid: "ca2f0c3dd5b0431a913c99ba5bfd2bd5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb92e98ad74b42af916785483c36e382" is_pre_election: true
I20260812 21:03:39.851621 15670 raft_consensus.cc:2454] T f3466ece79414d0ebd1144531d0ddd25 P 92601044a2eb4651b9d3bb09cbe9d5b0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca2f0c3dd5b0431a913c99ba5bfd2bd5 in term 0.
I20260812 21:03:39.851648 16205 raft_consensus.cc:2454] T f3466ece79414d0ebd1144531d0ddd25 P cb92e98ad74b42af916785483c36e382 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca2f0c3dd5b0431a913c99ba5bfd2bd5 in term 0.
I20260812 21:03:39.851850 15807 leader_election.cc:302] T f3466ece79414d0ebd1144531d0ddd25 P ca2f0c3dd5b0431a913c99ba5bfd2bd5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92601044a2eb4651b9d3bb09cbe9d5b0, ca2f0c3dd5b0431a913c99ba5bfd2bd5; no voters:
I20260812 21:03:39.851961 16532 raft_consensus.cc:2781] T f3466ece79414d0ebd1144531d0ddd25 P ca2f0c3dd5b0431a913c99ba5bfd2bd5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 21:03:39.851998 16532 raft_consensus.cc:482] T f3466ece79414d0ebd1144531d0ddd25 P ca2f0c3dd5b0431a913c99ba5bfd2bd5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 21:03:39.852034 16532 raft_consensus.cc:3037] T f3466ece79414d0ebd1144531d0ddd25 P ca2f0c3dd5b0431a913c99ba5bfd2bd5 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:03:39.853701 16532 raft_consensus.cc:504] T f3466ece79414d0ebd1144531d0ddd25 P ca2f0c3dd5b0431a913c99ba5bfd2bd5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92601044a2eb4651b9d3bb09cbe9d5b0" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 33109 } } peers { permanent_uuid: "ca2f0c3dd5b0431a913c99ba5bfd2bd5" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 44949 } } peers { permanent_uuid: "cb92e98ad74b42af916785483c36e382" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 35785 } }
I20260812 21:03:39.853866 16532 leader_election.cc:288] T f3466ece79414d0ebd1144531d0ddd25 P ca2f0c3dd5b0431a913c99ba5bfd2bd5 [CANDIDATE]: Term 1 election: Requested vote from peers 92601044a2eb4651b9d3bb09cbe9d5b0 (127.11.254.129:33109), cb92e98ad74b42af916785483c36e382 (127.11.254.131:35785)
I20260812 21:03:39.854069 16205 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f3466ece79414d0ebd1144531d0ddd25" candidate_uuid: "ca2f0c3dd5b0431a913c99ba5bfd2bd5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb92e98ad74b42af916785483c36e382"
I20260812 21:03:39.854164 16205 raft_consensus.cc:3037] T f3466ece79414d0ebd1144531d0ddd25 P cb92e98ad74b42af916785483c36e382 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:03:39.854148 15670 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f3466ece79414d0ebd1144531d0ddd25" candidate_uuid: "ca2f0c3dd5b0431a913c99ba5bfd2bd5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92601044a2eb4651b9d3bb09cbe9d5b0"
I20260812 21:03:39.854221 15670 raft_consensus.cc:3037] T f3466ece79414d0ebd1144531d0ddd25 P 92601044a2eb4651b9d3bb09cbe9d5b0 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:03:39.854983 15670 raft_consensus.cc:2454] T f3466ece79414d0ebd1144531d0ddd25 P 92601044a2eb4651b9d3bb09cbe9d5b0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca2f0c3dd5b0431a913c99ba5bfd2bd5 in term 1.
I20260812 21:03:39.854983 16205 raft_consensus.cc:2454] T f3466ece79414d0ebd1144531d0ddd25 P cb92e98ad74b42af916785483c36e382 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca2f0c3dd5b0431a913c99ba5bfd2bd5 in term 1.
I20260812 21:03:39.855185 15811 leader_election.cc:302] T f3466ece79414d0ebd1144531d0ddd25 P ca2f0c3dd5b0431a913c99ba5bfd2bd5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ca2f0c3dd5b0431a913c99ba5bfd2bd5, cb92e98ad74b42af916785483c36e382; no voters:
I20260812 21:03:39.855301 16532 raft_consensus.cc:2781] T f3466ece79414d0ebd1144531d0ddd25 P ca2f0c3dd5b0431a913c99ba5bfd2bd5 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:03:39.855355 16532 raft_consensus.cc:686] T f3466ece79414d0ebd1144531d0ddd25 P ca2f0c3dd5b0431a913c99ba5bfd2bd5 [term 1 LEADER]: Becoming Leader. State: Replica: ca2f0c3dd5b0431a913c99ba5bfd2bd5, State: Running, Role: LEADER
I20260812 21:03:39.855412 16532 consensus_queue.cc:237] T f3466ece79414d0ebd1144531d0ddd25 P ca2f0c3dd5b0431a913c99ba5bfd2bd5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92601044a2eb4651b9d3bb09cbe9d5b0" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 33109 } } peers { permanent_uuid: "ca2f0c3dd5b0431a913c99ba5bfd2bd5" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 44949 } } peers { permanent_uuid: "cb92e98ad74b42af916785483c36e382" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 35785 } }
I20260812 21:03:39.856025 15355 catalog_manager.cc:5495] T f3466ece79414d0ebd1144531d0ddd25 P ca2f0c3dd5b0431a913c99ba5bfd2bd5 reported cstate change: term changed from 0 to 1, leader changed from <none> to ca2f0c3dd5b0431a913c99ba5bfd2bd5 (127.11.254.130). New cstate: current_term: 1 leader_uuid: "ca2f0c3dd5b0431a913c99ba5bfd2bd5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92601044a2eb4651b9d3bb09cbe9d5b0" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 33109 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ca2f0c3dd5b0431a913c99ba5bfd2bd5" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 44949 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb92e98ad74b42af916785483c36e382" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 35785 } health_report { overall_health: UNKNOWN } } }
I20260812 21:03:39.862825 16205 raft_consensus.cc:1261] T f3466ece79414d0ebd1144531d0ddd25 P cb92e98ad74b42af916785483c36e382 [term 1 FOLLOWER]: Refusing update from remote peer ca2f0c3dd5b0431a913c99ba5bfd2bd5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 21:03:39.862972 15670 raft_consensus.cc:1261] T f3466ece79414d0ebd1144531d0ddd25 P 92601044a2eb4651b9d3bb09cbe9d5b0 [term 1 FOLLOWER]: Refusing update from remote peer ca2f0c3dd5b0431a913c99ba5bfd2bd5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 21:03:39.863035 16549 consensus_queue.cc:1035] T f3466ece79414d0ebd1144531d0ddd25 P ca2f0c3dd5b0431a913c99ba5bfd2bd5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb92e98ad74b42af916785483c36e382" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 35785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 21:03:39.863173 16549 consensus_queue.cc:1035] T f3466ece79414d0ebd1144531d0ddd25 P ca2f0c3dd5b0431a913c99ba5bfd2bd5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92601044a2eb4651b9d3bb09cbe9d5b0" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 33109 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260812 21:03:39.868162 16453 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260812 21:03:41.386083 15361 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53836:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 21:03:41.386191 15361 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53836:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 21:03:41.387010 15361 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f41525e3e1134a4887fe0909de2f21f6: Sending DeleteTablet for 3 replicas of tablet f3466ece79414d0ebd1144531d0ddd25
I20260812 21:03:41.387251 15630 tablet_service.cc:1514] Processing DeleteTablet for tablet f3466ece79414d0ebd1144531d0ddd25 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:03:41 UTC) from {username='kudu', principal='kudu/127.11.254.190@KRBTEST.COM'} at 127.0.0.1:40914
I20260812 21:03:41.387328 15906 tablet_service.cc:1514] Processing DeleteTablet for tablet f3466ece79414d0ebd1144531d0ddd25 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:03:41 UTC) from {username='kudu', principal='kudu/127.11.254.190@KRBTEST.COM'} at 127.0.0.1:53566
I20260812 21:03:41.387457 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 15503
I20260812 21:03:41.387523 16607 tablet_replica.cc:330] T f3466ece79414d0ebd1144531d0ddd25 P ca2f0c3dd5b0431a913c99ba5bfd2bd5: stopping tablet replica
I20260812 21:03:41.388031 16607 raft_consensus.cc:2229] T f3466ece79414d0ebd1144531d0ddd25 P ca2f0c3dd5b0431a913c99ba5bfd2bd5 [term 1 LEADER]: Raft consensus shutting down.
I20260812 21:03:41.388115 16607 raft_consensus.cc:2258] T f3466ece79414d0ebd1144531d0ddd25 P ca2f0c3dd5b0431a913c99ba5bfd2bd5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 21:03:41.398487 16607 ts_tablet_manager.cc:1905] T f3466ece79414d0ebd1144531d0ddd25 P ca2f0c3dd5b0431a913c99ba5bfd2bd5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 21:03:41.387943 16169 tablet_service.cc:1514] Processing DeleteTablet for tablet f3466ece79414d0ebd1144531d0ddd25 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:03:41 UTC) from {username='kudu', principal='kudu/127.11.254.190@KRBTEST.COM'} at 127.0.0.1:57404
I20260812 21:03:41.400231 16611 tablet_replica.cc:330] T f3466ece79414d0ebd1144531d0ddd25 P cb92e98ad74b42af916785483c36e382: stopping tablet replica
I20260812 21:03:41.400315 16611 raft_consensus.cc:2229] T f3466ece79414d0ebd1144531d0ddd25 P cb92e98ad74b42af916785483c36e382 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 21:03:41.400370 16611 raft_consensus.cc:2258] T f3466ece79414d0ebd1144531d0ddd25 P cb92e98ad74b42af916785483c36e382 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 21:03:41.400657 16611 ts_tablet_manager.cc:1905] T f3466ece79414d0ebd1144531d0ddd25 P cb92e98ad74b42af916785483c36e382: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 21:03:41.400803 16607 ts_tablet_manager.cc:1918] T f3466ece79414d0ebd1144531d0ddd25 P ca2f0c3dd5b0431a913c99ba5bfd2bd5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 21:03:41.400866 16607 log.cc:1196] T f3466ece79414d0ebd1144531d0ddd25 P ca2f0c3dd5b0431a913c99ba5bfd2bd5: Deleting WAL directory at /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568607354921-12282-0/minicluster-data/ts-1/wal/wals/f3466ece79414d0ebd1144531d0ddd25
I20260812 21:03:41.401098 16607 ts_tablet_manager.cc:1939] T f3466ece79414d0ebd1144531d0ddd25 P ca2f0c3dd5b0431a913c99ba5bfd2bd5: Deleting consensus metadata
I20260812 21:03:41.401639 15338 catalog_manager.cc:4841] TS ca2f0c3dd5b0431a913c99ba5bfd2bd5 (127.11.254.130:44949): tablet f3466ece79414d0ebd1144531d0ddd25 (table test-table [id=96e01394bb8c4585a399e855e99ffe5a]) successfully deleted
I20260812 21:03:41.402200 16611 ts_tablet_manager.cc:1918] T f3466ece79414d0ebd1144531d0ddd25 P cb92e98ad74b42af916785483c36e382: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 21:03:41.402251 16611 log.cc:1196] T f3466ece79414d0ebd1144531d0ddd25 P cb92e98ad74b42af916785483c36e382: Deleting WAL directory at /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786568607354921-12282-0/minicluster-data/ts-2/wal/wals/f3466ece79414d0ebd1144531d0ddd25
I20260812 21:03:41.402417 16611 ts_tablet_manager.cc:1939] T f3466ece79414d0ebd1144531d0ddd25 P cb92e98ad74b42af916785483c36e382: Deleting consensus metadata
I20260812 21:03:41.402848 15338 catalog_manager.cc:4841] TS cb92e98ad74b42af916785483c36e382 (127.11.254.131:35785): tablet f3466ece79414d0ebd1144531d0ddd25 (table test-table [id=96e01394bb8c4585a399e855e99ffe5a]) successfully deleted
W20260812 21:03:41.405793 15334 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.254.129:33109 (error 108)
I20260812 21:03:41.406095 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 15778
I20260812 21:03:41.412252 16617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:41.406201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.129:33109 (local address 127.0.0.1:44670)
0812 21:03:41.412104 (+ 5903us) negotiation.cc:107] Waiting for socket to connect
0812 21:03:41.412196 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.129:33109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":5864,"thread_start_us":66,"threads_started":1}
W20260812 21:03:41.412387 15334 catalog_manager.cc:4584] TS 92601044a2eb4651b9d3bb09cbe9d5b0 (127.11.254.129:33109): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f3466ece79414d0ebd1144531d0ddd25: Network error: Client connection negotiation failed: client connection to 127.11.254.129:33109: connect: Connection refused (error 111)
I20260812 21:03:41.419409 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 16045
I20260812 21:03:41.430711 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 15301
2026-08-12T21:03:41Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6052 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568607354921-12282-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 21:03:41 dist-test-slave-vpvm krb5kdc[16621](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 21:03:41 dist-test-slave-vpvm krb5kdc[16621](info): set up 2 sockets
Aug 12 21:03:41 dist-test-slave-vpvm krb5kdc[16621](info): commencing operation
krb5kdc: starting...
W20260812 21:03:42.615285 12282 mini_kdc.cc:121] Time spent starting KDC: real 1.153s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568607354921-12282-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568607354921-12282-0/krb5kdc/test-user.keytab.
Aug 12 21:03:42 dist-test-slave-vpvm krb5kdc[16621](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568622, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T21:03:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T21:03:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.254.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.254.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.254.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.190.keytab.
Entry for principal kudu/127.11.254.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.190.keytab.
I20260812 21:03:42.739063 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568607354921-12282-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568607354921-12282-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568607354921-12282-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.254.190:45795
--webserver_interface=127.11.254.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.190.keytab
--principal=kudu/127.11.254.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.254.148:36819
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.254.190:45795
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568607354921-12282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568607354921-12282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568607354921-12282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:03:42.821249 16649 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:03:42.821475 16649 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:03:42.821508 16649 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:03:42.822676 16649 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 21:03:42.822721 16649 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:03:42.822739 16649 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:03:42.822755 16649 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 21:03:42.822770 16649 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 21:03:42.824304 16649 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.254.148:36819
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568607354921-12282-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568607354921-12282-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.254.190:45795
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.190.keytab
--principal=kudu/127.11.254.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.254.190:45795
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.254.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16649
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568607354921-12282-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:03:42.824559 16649 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:03:42.824779 16649 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:03:42.829692 16661 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:03:42.829721 16658 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:03:42.829736 16657 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:03:42.829885 16649 server_base.cc:1034] running on GCE node
I20260812 21:03:42.830144 16649 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:03:42.830399 16649 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:03:42.831568 16649 hybrid_clock.cc:648] HybridClock initialized: now 1786568622831552 us; error 30 us; skew 500 ppm
Aug 12 21:03:42 dist-test-slave-vpvm krb5kdc[16621](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568622, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:03:42.834112 16649 init.cc:377] Logged in from keytab as kudu/127.11.254.190@KRBTEST.COM (short username kudu)
I20260812 21:03:42.834563 16649 webserver.cc:460] Webserver started at http://127.11.254.190:34091/ using document root <none> and password file <none>
I20260812 21:03:42.834789 16649 fs_manager.cc:359] Metadata directory not provided
I20260812 21:03:42.834832 16649 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:03:42.834957 16649 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:03:42.836017 16649 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568607354921-12282-0/minicluster-data/master-0/data/instance:
uuid: "0f7eac8b6c2d416f9dd7f1e996e51e9c"
format_stamp: "Formatted at 2026-08-12 21:03:42 on dist-test-slave-vpvm"
server_key: "1b1c51be5051a2c8e838953dd63a4b97"
server_key_iv: "358232c68bf549f85faa87cafc54e545"
server_key_version: "encryptionkey@0"
I20260812 21:03:42.836380 16649 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568607354921-12282-0/minicluster-data/master-0/wal/instance:
uuid: "0f7eac8b6c2d416f9dd7f1e996e51e9c"
format_stamp: "Formatted at 2026-08-12 21:03:42 on dist-test-slave-vpvm"
server_key: "1b1c51be5051a2c8e838953dd63a4b97"
server_key_iv: "358232c68bf549f85faa87cafc54e545"
server_key_version: "encryptionkey@0"
I20260812 21:03:42.837648 16649 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 21:03:42.838367 16678 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:03:42.838521 16649 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 21:03:42.838609 16649 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568607354921-12282-0/minicluster-data/master-0/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568607354921-12282-0/minicluster-data/master-0/wal
uuid: "0f7eac8b6c2d416f9dd7f1e996e51e9c"
format_stamp: "Formatted at 2026-08-12 21:03:42 on dist-test-slave-vpvm"
server_key: "1b1c51be5051a2c8e838953dd63a4b97"
server_key_iv: "358232c68bf549f85faa87cafc54e545"
server_key_version: "encryptionkey@0"
I20260812 21:03:42.838675 16649 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568607354921-12282-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568607354921-12282-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568607354921-12282-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:03:42.851954 16649 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:03:42.853693 16649 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:03:42.853832 16649 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:03:42.857836 16649 rpc_server.cc:307] RPC server started. Bound to: 127.11.254.190:45795
I20260812 21:03:42.857870 16775 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.254.190:45795 every 8 connection(s)
I20260812 21:03:42.858242 16649 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
I20260812 21:03:42.859124 16777 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:03:42.861449 16777 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0f7eac8b6c2d416f9dd7f1e996e51e9c: Bootstrap starting.
I20260812 21:03:42.862041 16777 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0f7eac8b6c2d416f9dd7f1e996e51e9c: Neither blocks nor log segments found. Creating new log.
I20260812 21:03:42.862280 16777 log.cc:824] T 00000000000000000000000000000000 P 0f7eac8b6c2d416f9dd7f1e996e51e9c: Log is configured to *not* fsync() on all Append() calls
I20260812 21:03:42.862926 16777 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0f7eac8b6c2d416f9dd7f1e996e51e9c: No bootstrap required, opened a new log
I20260812 21:03:42.864462 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 16649
I20260812 21:03:42.864547 12282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568607354921-12282-0/minicluster-data/master-0/wal/instance
I20260812 21:03:42.864699 12282 external_mini_cluster.cc:1428] Setting key 31367b947a7b88e2c212bf17fc1061bd
I20260812 21:03:42.864707 16777 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0f7eac8b6c2d416f9dd7f1e996e51e9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f7eac8b6c2d416f9dd7f1e996e51e9c" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 45795 } }
I20260812 21:03:42.864874 16777 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0f7eac8b6c2d416f9dd7f1e996e51e9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:03:42.864912 16777 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0f7eac8b6c2d416f9dd7f1e996e51e9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f7eac8b6c2d416f9dd7f1e996e51e9c, State: Initialized, Role: FOLLOWER
I20260812 21:03:42.865065 16777 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0f7eac8b6c2d416f9dd7f1e996e51e9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f7eac8b6c2d416f9dd7f1e996e51e9c" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 45795 } }
I20260812 21:03:42.865160 16777 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0f7eac8b6c2d416f9dd7f1e996e51e9c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 21:03:42.865195 16777 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0f7eac8b6c2d416f9dd7f1e996e51e9c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 21:03:42.865242 16777 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0f7eac8b6c2d416f9dd7f1e996e51e9c [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:03:42.866315 16777 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0f7eac8b6c2d416f9dd7f1e996e51e9c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f7eac8b6c2d416f9dd7f1e996e51e9c" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 45795 } }
I20260812 21:03:42.866421 16777 leader_election.cc:302] T 00000000000000000000000000000000 P 0f7eac8b6c2d416f9dd7f1e996e51e9c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0f7eac8b6c2d416f9dd7f1e996e51e9c; no voters:
I20260812 21:03:42.866583 16777 leader_election.cc:288] T 00000000000000000000000000000000 P 0f7eac8b6c2d416f9dd7f1e996e51e9c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 21:03:42.866648 16784 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0f7eac8b6c2d416f9dd7f1e996e51e9c [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:03:42.866797 16784 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0f7eac8b6c2d416f9dd7f1e996e51e9c [term 1 LEADER]: Becoming Leader. State: Replica: 0f7eac8b6c2d416f9dd7f1e996e51e9c, State: Running, Role: LEADER
I20260812 21:03:42.866844 16777 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0f7eac8b6c2d416f9dd7f1e996e51e9c [sys.catalog]: configured and running, proceeding with master startup.
I20260812 21:03:42.866888 16784 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0f7eac8b6c2d416f9dd7f1e996e51e9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f7eac8b6c2d416f9dd7f1e996e51e9c" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 45795 } }
I20260812 21:03:42.867302 16784 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0f7eac8b6c2d416f9dd7f1e996e51e9c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0f7eac8b6c2d416f9dd7f1e996e51e9c. Latest consensus state: current_term: 1 leader_uuid: "0f7eac8b6c2d416f9dd7f1e996e51e9c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f7eac8b6c2d416f9dd7f1e996e51e9c" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 45795 } } }
I20260812 21:03:42.867395 16784 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0f7eac8b6c2d416f9dd7f1e996e51e9c [sys.catalog]: This master's current role is: LEADER
I20260812 21:03:42.867311 16786 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0f7eac8b6c2d416f9dd7f1e996e51e9c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0f7eac8b6c2d416f9dd7f1e996e51e9c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f7eac8b6c2d416f9dd7f1e996e51e9c" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 45795 } } }
I20260812 21:03:42.867774 16786 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0f7eac8b6c2d416f9dd7f1e996e51e9c [sys.catalog]: This master's current role is: LEADER
I20260812 21:03:42.867859 16792 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 21:03:42.868357 16792 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 21:03:42.869931 16792 catalog_manager.cc:1347] Generated new cluster ID: a77856f5b64249ba8fbe8ff9f5b9c398
I20260812 21:03:42.870002 16792 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 21:03:42 dist-test-slave-vpvm krb5kdc[16621](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568622, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.254.190@KRBTEST.COM
I20260812 21:03:42.876107 16783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:42.865163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41424 (local address 127.11.254.190:45795)
0812 21:03:42.865382 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0812 21:03:42.865393 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 21:03:42.865428 (+ 35us) server_negotiation.cc:407] Connection header received
0812 21:03:42.865556 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:03:42.865564 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:03:42.865650 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:03:42.865822 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:03:42.866479 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:42.868100 (+ 1621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:42.868907 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:42.869526 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:42.869649 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:42.872505 (+ 2856us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:03:42.872510 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:03:42.872510 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:03:42.872523 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:03:42.874442 (+ 1919us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:42.874842 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:42.874844 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:42.874844 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:42.874962 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:42.875183 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:42.875183 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:42.875184 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:42.875492 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:03:42.875632 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0812 21:03:42.875763 (+ 131us) server_negotiation.cc:299] Negotiation successful
0812 21:03:42.875904 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":105,"threads_started":1}
I20260812 21:03:42.881433 16792 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 21:03:42.881968 16792 catalog_manager.cc:1504] Loading token signing keys...
I20260812 21:03:42.885089 16792 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0f7eac8b6c2d416f9dd7f1e996e51e9c: Generated new TSK 0
I20260812 21:03:42.885267 16792 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.254.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.254.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.254.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.129.keytab.
Entry for principal kudu/127.11.254.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.129.keytab.
I20260812 21:03:42.912971 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568607354921-12282-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568607354921-12282-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568607354921-12282-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568607354921-12282-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.254.129:0
--local_ip_for_outbound_sockets=127.11.254.129
--webserver_interface=127.11.254.129
--webserver_port=0
--tserver_master_addrs=127.11.254.190:45795
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.129.keytab
--principal=kudu/127.11.254.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.254.148:36819
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568607354921-12282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568607354921-12282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568607354921-12282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:03:42.997684 16818 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:03:42.997905 16818 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:03:42.997931 16818 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:03:42.999198 16818 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:03:42.999246 16818 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:03:42.999280 16818 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.254.129
I20260812 21:03:43.000777 16818 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.254.148:36819
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568607354921-12282-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568607354921-12282-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.129.keytab
--principal=kudu/127.11.254.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.254.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568607354921-12282-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.254.129
--webserver_port=0
--tserver_master_addrs=127.11.254.190:45795
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16818
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.254.129
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568607354921-12282-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:03:43.001008 16818 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:03:43.001250 16818 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:03:43.006237 16831 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:03:43.006319 16829 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:03:43.006469 16818 server_base.cc:1034] running on GCE node
W20260812 21:03:43.006304 16827 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:03:43.006713 16818 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:03:43.006917 16818 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:03:43.008052 16818 hybrid_clock.cc:648] HybridClock initialized: now 1786568623008035 us; error 32 us; skew 500 ppm
Aug 12 21:03:43 dist-test-slave-vpvm krb5kdc[16621](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568623, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:03:43.010529 16818 init.cc:377] Logged in from keytab as kudu/127.11.254.129@KRBTEST.COM (short username kudu)
I20260812 21:03:43.010989 16818 webserver.cc:460] Webserver started at http://127.11.254.129:46159/ using document root <none> and password file <none>
I20260812 21:03:43.011240 16818 fs_manager.cc:359] Metadata directory not provided
I20260812 21:03:43.011286 16818 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:03:43.011415 16818 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:03:43.012526 16818 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568607354921-12282-0/minicluster-data/ts-0/data/instance:
uuid: "2695553e898a4c52b35c92daf35defb2"
format_stamp: "Formatted at 2026-08-12 21:03:43 on dist-test-slave-vpvm"
server_key: "3df375fb665f5ac3bad770b48cc30943"
server_key_iv: "87b09d850a58684551cf4cb7470e943b"
server_key_version: "encryptionkey@0"
I20260812 21:03:43.012869 16818 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568607354921-12282-0/minicluster-data/ts-0/wal/instance:
uuid: "2695553e898a4c52b35c92daf35defb2"
format_stamp: "Formatted at 2026-08-12 21:03:43 on dist-test-slave-vpvm"
server_key: "3df375fb665f5ac3bad770b48cc30943"
server_key_iv: "87b09d850a58684551cf4cb7470e943b"
server_key_version: "encryptionkey@0"
I20260812 21:03:43.014210 16818 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260812 21:03:43.014921 16850 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:03:43.015064 16818 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 21:03:43.015144 16818 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568607354921-12282-0/minicluster-data/ts-0/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568607354921-12282-0/minicluster-data/ts-0/wal
uuid: "2695553e898a4c52b35c92daf35defb2"
format_stamp: "Formatted at 2026-08-12 21:03:43 on dist-test-slave-vpvm"
server_key: "3df375fb665f5ac3bad770b48cc30943"
server_key_iv: "87b09d850a58684551cf4cb7470e943b"
server_key_version: "encryptionkey@0"
I20260812 21:03:43.015215 16818 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568607354921-12282-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568607354921-12282-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568607354921-12282-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:03:43.027056 16818 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:03:43.028920 16818 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:03:43.029057 16818 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:03:43.029313 16818 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:03:43.029623 16818 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:03:43.029657 16818 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:03:43.029681 16818 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:03:43.029706 16818 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:03:43.035516 16818 rpc_server.cc:307] RPC server started. Bound to: 127.11.254.129:33825
I20260812 21:03:43.035573 17071 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.254.129:33825 every 8 connection(s)
I20260812 21:03:43.035889 16818 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568607354921-12282-0/minicluster-data/ts-0/data/info.pb
I20260812 21:03:43.038926 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 16818
I20260812 21:03:43.039027 12282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568607354921-12282-0/minicluster-data/ts-0/wal/instance
I20260812 21:03:43.039212 12282 external_mini_cluster.cc:1428] Setting key 17d95fd14c7570e990fd5a9ea6e92369
Aug 12 21:03:43 dist-test-slave-vpvm krb5kdc[16621](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568623, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.129@KRBTEST.COM for kudu/127.11.254.190@KRBTEST.COM
I20260812 21:03:43.045594 16783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:43.036576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.129:57325 (local address 127.11.254.190:45795)
0812 21:03:43.036667 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0812 21:03:43.036669 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:03:43.036800 (+ 131us) server_negotiation.cc:407] Connection header received
0812 21:03:43.037296 (+ 496us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:03:43.037297 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:03:43.037311 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:03:43.037358 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:03:43.038355 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:43.038869 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:43.040203 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:43.040404 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:43.040457 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:43.042822 (+ 2365us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:03:43.042824 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:03:43.042825 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:03:43.042833 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:03:43.044239 (+ 1406us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:43.044658 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:43.044660 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:43.044660 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:43.044727 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:43.044938 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:43.044939 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:43.044939 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:43.045088 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:03:43.045157 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0812 21:03:43.045391 (+ 234us) server_negotiation.cc:299] Negotiation successful
0812 21:03:43.045495 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 21:03:43.045653 17077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:43.036533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:45795 (local address 127.11.254.129:57325)
0812 21:03:43.036678 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0812 21:03:43.036715 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0812 21:03:43.037213 (+ 498us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:03:43.037423 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:03:43.037426 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:03:43.038057 (+ 631us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:03:43.038272 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:43.038279 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:43.038965 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:43.038966 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:43.040123 (+ 1157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:43.040126 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:43.040483 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:43.040484 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:43.040631 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:43.041155 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:03:43.041165 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:03:43.042728 (+ 1563us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:03:43.044321 (+ 1593us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:03:43.044329 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:03:43.044331 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:03:43.044568 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:03:43.044787 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:03:43.044788 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:03:43.044788 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:03:43.044886 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:03:43.045144 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:03:43.045147 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:03:43.045266 (+ 119us) client_negotiation.cc:769] Sending connection context
0812 21:03:43.045344 (+ 78us) client_negotiation.cc:240] Negotiation successful
0812 21:03:43.045474 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":78,"threads_started":1}
I20260812 21:03:43.046186 17073 heartbeater.cc:347] Connected to a master server at 127.11.254.190:45795
I20260812 21:03:43.046283 17073 heartbeater.cc:464] Registering TS with master...
I20260812 21:03:43.046480 17073 heartbeater.cc:511] Master 127.11.254.190:45795 requested a full tablet report, sending...
I20260812 21:03:43.047004 16709 ts_manager.cc:194] Registered new tserver with Master: 2695553e898a4c52b35c92daf35defb2 (127.11.254.129:33825)
I20260812 21:03:43.047626 16709 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.254.129@KRBTEST.COM'} at 127.11.254.129:57325
WARNING: no policy specified for kudu/127.11.254.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.254.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.254.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.130.keytab.
Entry for principal kudu/127.11.254.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.130.keytab.
I20260812 21:03:43.066344 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568607354921-12282-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568607354921-12282-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568607354921-12282-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568607354921-12282-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.254.130:0
--local_ip_for_outbound_sockets=127.11.254.130
--webserver_interface=127.11.254.130
--webserver_port=0
--tserver_master_addrs=127.11.254.190:45795
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.130.keytab
--principal=kudu/127.11.254.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.254.148:36819
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568607354921-12282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568607354921-12282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568607354921-12282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:03:43.150177 17085 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:03:43.150400 17085 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:03:43.150431 17085 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:03:43.151618 17085 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:03:43.151664 17085 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:03:43.151701 17085 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.254.130
I20260812 21:03:43.153160 17085 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.254.148:36819
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568607354921-12282-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568607354921-12282-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.130.keytab
--principal=kudu/127.11.254.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.254.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568607354921-12282-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.254.130
--webserver_port=0
--tserver_master_addrs=127.11.254.190:45795
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17085
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.254.130
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568607354921-12282-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:03:43.153409 17085 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:03:43.153635 17085 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:03:43.158738 17094 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:03:43.158792 17085 server_base.cc:1034] running on GCE node
W20260812 21:03:43.158731 17091 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:03:43.158731 17092 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:03:43.159140 17085 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:03:43.159391 17085 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:03:43.160554 17085 hybrid_clock.cc:648] HybridClock initialized: now 1786568623160536 us; error 30 us; skew 500 ppm
Aug 12 21:03:43 dist-test-slave-vpvm krb5kdc[16621](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568623, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:03:43.163180 17085 init.cc:377] Logged in from keytab as kudu/127.11.254.130@KRBTEST.COM (short username kudu)
I20260812 21:03:43.163614 17085 webserver.cc:460] Webserver started at http://127.11.254.130:42705/ using document root <none> and password file <none>
I20260812 21:03:43.163820 17085 fs_manager.cc:359] Metadata directory not provided
I20260812 21:03:43.163870 17085 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:03:43.163995 17085 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:03:43.165071 17085 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568607354921-12282-0/minicluster-data/ts-1/data/instance:
uuid: "0cd3c53b40154a1faf2317ddf5d8ad2d"
format_stamp: "Formatted at 2026-08-12 21:03:43 on dist-test-slave-vpvm"
server_key: "1ec3c4bc6ec485d5ade4e63d78a78d1f"
server_key_iv: "3d387cc76c7b0b127c9765530b3398ac"
server_key_version: "encryptionkey@0"
I20260812 21:03:43.165429 17085 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568607354921-12282-0/minicluster-data/ts-1/wal/instance:
uuid: "0cd3c53b40154a1faf2317ddf5d8ad2d"
format_stamp: "Formatted at 2026-08-12 21:03:43 on dist-test-slave-vpvm"
server_key: "1ec3c4bc6ec485d5ade4e63d78a78d1f"
server_key_iv: "3d387cc76c7b0b127c9765530b3398ac"
server_key_version: "encryptionkey@0"
I20260812 21:03:43.166797 17085 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 21:03:43.167491 17104 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:03:43.167695 17085 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 21:03:43.167784 17085 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568607354921-12282-0/minicluster-data/ts-1/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568607354921-12282-0/minicluster-data/ts-1/wal
uuid: "0cd3c53b40154a1faf2317ddf5d8ad2d"
format_stamp: "Formatted at 2026-08-12 21:03:43 on dist-test-slave-vpvm"
server_key: "1ec3c4bc6ec485d5ade4e63d78a78d1f"
server_key_iv: "3d387cc76c7b0b127c9765530b3398ac"
server_key_version: "encryptionkey@0"
I20260812 21:03:43.167852 17085 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568607354921-12282-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568607354921-12282-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568607354921-12282-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:03:43.176314 17085 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:03:43.178537 17085 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:03:43.178686 17085 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:03:43.178927 17085 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:03:43.179235 17085 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:03:43.179270 17085 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:03:43.179301 17085 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:03:43.179318 17085 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:03:43.185083 17085 rpc_server.cc:307] RPC server started. Bound to: 127.11.254.130:38181
I20260812 21:03:43.185128 17326 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.254.130:38181 every 8 connection(s)
I20260812 21:03:43.185444 17085 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568607354921-12282-0/minicluster-data/ts-1/data/info.pb
Aug 12 21:03:43 dist-test-slave-vpvm krb5kdc[16621](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568623, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.130@KRBTEST.COM for kudu/127.11.254.190@KRBTEST.COM
I20260812 21:03:43.192297 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 17085
I20260812 21:03:43.192389 12282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568607354921-12282-0/minicluster-data/ts-1/wal/instance
I20260812 21:03:43.192590 12282 external_mini_cluster.cc:1428] Setting key 34e9ee9644eeafff87cecc17528da735
I20260812 21:03:43.195166 16783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:43.186301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.130:52869 (local address 127.11.254.190:45795)
0812 21:03:43.186371 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0812 21:03:43.186373 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:03:43.186532 (+ 159us) server_negotiation.cc:407] Connection header received
0812 21:03:43.187046 (+ 514us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:03:43.187047 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:03:43.187072 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:03:43.187132 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:03:43.188096 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:43.188605 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:43.189647 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:43.189896 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:43.189979 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:43.192152 (+ 2173us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:03:43.192154 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:03:43.192154 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:03:43.192161 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:03:43.193623 (+ 1462us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:43.194100 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:43.194101 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:43.194102 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:43.194177 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:43.194464 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:43.194465 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:43.194466 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:43.194594 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:03:43.194661 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 21:03:43.194982 (+ 321us) server_negotiation.cc:299] Negotiation successful
0812 21:03:43.195072 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260812 21:03:43.195503 17332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:43.186247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:45795 (local address 127.11.254.130:52869)
0812 21:03:43.186411 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0812 21:03:43.186453 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 21:03:43.186942 (+ 489us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:03:43.187201 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:03:43.187204 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:03:43.187795 (+ 591us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:03:43.188019 (+ 224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:43.188025 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:43.188688 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:43.188689 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:43.189572 (+ 883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:43.189575 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:43.190001 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:43.190003 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:43.190162 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:43.190676 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:03:43.190684 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:03:43.192058 (+ 1374us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:03:43.193711 (+ 1653us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:03:43.193718 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:03:43.193721 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:03:43.194030 (+ 309us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:03:43.194243 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:03:43.194243 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:03:43.194244 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:03:43.194372 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:03:43.194661 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:03:43.194666 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:03:43.194837 (+ 171us) client_negotiation.cc:769] Sending connection context
0812 21:03:43.194947 (+ 110us) client_negotiation.cc:240] Negotiation successful
0812 21:03:43.195128 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":90,"threads_started":1}
I20260812 21:03:43.195936 17327 heartbeater.cc:347] Connected to a master server at 127.11.254.190:45795
I20260812 21:03:43.196029 17327 heartbeater.cc:464] Registering TS with master...
I20260812 21:03:43.196216 17327 heartbeater.cc:511] Master 127.11.254.190:45795 requested a full tablet report, sending...
I20260812 21:03:43.196580 16709 ts_manager.cc:194] Registered new tserver with Master: 0cd3c53b40154a1faf2317ddf5d8ad2d (127.11.254.130:38181)
I20260812 21:03:43.196954 16709 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.254.130@KRBTEST.COM'} at 127.11.254.130:52869
WARNING: no policy specified for kudu/127.11.254.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.254.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.254.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.131.keytab.
Entry for principal kudu/127.11.254.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.131.keytab.
I20260812 21:03:43.220979 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568607354921-12282-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568607354921-12282-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568607354921-12282-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568607354921-12282-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.254.131:0
--local_ip_for_outbound_sockets=127.11.254.131
--webserver_interface=127.11.254.131
--webserver_port=0
--tserver_master_addrs=127.11.254.190:45795
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.131.keytab
--principal=kudu/127.11.254.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.254.148:36819
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568607354921-12282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568607354921-12282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568607354921-12282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:03:43.304051 17335 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:03:43.304298 17335 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:03:43.304323 17335 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:03:43.305557 17335 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:03:43.305604 17335 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:03:43.305637 17335 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.254.131
I20260812 21:03:43.307256 17335 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.254.148:36819
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568607354921-12282-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568607354921-12282-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.131.keytab
--principal=kudu/127.11.254.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.254.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568607354921-12282-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.254.131
--webserver_port=0
--tserver_master_addrs=127.11.254.190:45795
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17335
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.254.131
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568607354921-12282-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:03:43.307512 17335 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:03:43.307753 17335 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:03:43.312856 17343 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:03:43.312906 17346 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:03:43.313038 17348 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:03:43.313239 17335 server_base.cc:1034] running on GCE node
I20260812 21:03:43.313455 17335 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:03:43.313705 17335 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:03:43.314854 17335 hybrid_clock.cc:648] HybridClock initialized: now 1786568623314835 us; error 39 us; skew 500 ppm
Aug 12 21:03:43 dist-test-slave-vpvm krb5kdc[16621](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568623, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:03:43.317520 17335 init.cc:377] Logged in from keytab as kudu/127.11.254.131@KRBTEST.COM (short username kudu)
I20260812 21:03:43.318037 17335 webserver.cc:460] Webserver started at http://127.11.254.131:36983/ using document root <none> and password file <none>
I20260812 21:03:43.318276 17335 fs_manager.cc:359] Metadata directory not provided
I20260812 21:03:43.318326 17335 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:03:43.318459 17335 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:03:43.319576 17335 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568607354921-12282-0/minicluster-data/ts-2/data/instance:
uuid: "0065827d12df4186936238d649da83db"
format_stamp: "Formatted at 2026-08-12 21:03:43 on dist-test-slave-vpvm"
server_key: "5a78d38c9659aa8e6b8939035f0e21d5"
server_key_iv: "1c1e6cb989794220c3a0f84b6b9de96a"
server_key_version: "encryptionkey@0"
I20260812 21:03:43.319942 17335 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568607354921-12282-0/minicluster-data/ts-2/wal/instance:
uuid: "0065827d12df4186936238d649da83db"
format_stamp: "Formatted at 2026-08-12 21:03:43 on dist-test-slave-vpvm"
server_key: "5a78d38c9659aa8e6b8939035f0e21d5"
server_key_iv: "1c1e6cb989794220c3a0f84b6b9de96a"
server_key_version: "encryptionkey@0"
I20260812 21:03:43.321357 17335 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 21:03:43.322237 17360 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:03:43.322391 17335 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 21:03:43.322543 17335 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568607354921-12282-0/minicluster-data/ts-2/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568607354921-12282-0/minicluster-data/ts-2/wal
uuid: "0065827d12df4186936238d649da83db"
format_stamp: "Formatted at 2026-08-12 21:03:43 on dist-test-slave-vpvm"
server_key: "5a78d38c9659aa8e6b8939035f0e21d5"
server_key_iv: "1c1e6cb989794220c3a0f84b6b9de96a"
server_key_version: "encryptionkey@0"
I20260812 21:03:43.322615 17335 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568607354921-12282-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568607354921-12282-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568607354921-12282-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:03:43.364118 17335 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:03:43.366351 17335 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:03:43.366521 17335 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:03:43.366761 17335 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:03:43.367156 17335 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:03:43.367195 17335 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:03:43.367228 17335 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:03:43.367244 17335 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:03:43.373354 17335 rpc_server.cc:307] RPC server started. Bound to: 127.11.254.131:45353
I20260812 21:03:43.373401 17561 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.254.131:45353 every 8 connection(s)
I20260812 21:03:43.373739 17335 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568607354921-12282-0/minicluster-data/ts-2/data/info.pb
I20260812 21:03:43.377238 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 17335
I20260812 21:03:43.377326 12282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786568607354921-12282-0/minicluster-data/ts-2/wal/instance
I20260812 21:03:43.377480 12282 external_mini_cluster.cc:1428] Setting key 7052f9a6bc7380a441a3132975240bff
Aug 12 21:03:43 dist-test-slave-vpvm krb5kdc[16621](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568623, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.131@KRBTEST.COM for kudu/127.11.254.190@KRBTEST.COM
I20260812 21:03:43.383234 16783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:43.374502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.131:54573 (local address 127.11.254.190:45795)
0812 21:03:43.374574 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0812 21:03:43.374577 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 21:03:43.374774 (+ 197us) server_negotiation.cc:407] Connection header received
0812 21:03:43.375304 (+ 530us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:03:43.375305 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:03:43.375318 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:03:43.375372 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:03:43.376416 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:43.376946 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:43.378150 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:43.378436 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:43.378504 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:43.380642 (+ 2138us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:03:43.380643 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:03:43.380643 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:03:43.380651 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:03:43.381931 (+ 1280us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:43.382396 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:43.382396 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:43.382397 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:43.382447 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:43.382681 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:43.382682 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:43.382682 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:43.382794 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:03:43.382844 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0812 21:03:43.383086 (+ 242us) server_negotiation.cc:299] Negotiation successful
0812 21:03:43.383162 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260812 21:03:43.383443 17565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:43.374470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:45795 (local address 127.11.254.131:54573)
0812 21:03:43.374650 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0812 21:03:43.374689 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 21:03:43.375198 (+ 509us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:03:43.375445 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:03:43.375449 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:03:43.376077 (+ 628us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:03:43.376306 (+ 229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:43.376312 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:43.377043 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:43.377045 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:43.378078 (+ 1033us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:43.378081 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:43.378522 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:43.378523 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:43.378650 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:43.379080 (+ 430us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:03:43.379088 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:03:43.380547 (+ 1459us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:03:43.382074 (+ 1527us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:03:43.382082 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:03:43.382084 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:03:43.382328 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:03:43.382524 (+ 196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:03:43.382525 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:03:43.382525 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:03:43.382633 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:03:43.382842 (+ 209us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:03:43.382845 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:03:43.382981 (+ 136us) client_negotiation.cc:769] Sending connection context
0812 21:03:43.383067 (+ 86us) client_negotiation.cc:240] Negotiation successful
0812 21:03:43.383213 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":87,"threads_started":1}
I20260812 21:03:43.383913 17563 heartbeater.cc:347] Connected to a master server at 127.11.254.190:45795
I20260812 21:03:43.384019 17563 heartbeater.cc:464] Registering TS with master...
I20260812 21:03:43.384182 17563 heartbeater.cc:511] Master 127.11.254.190:45795 requested a full tablet report, sending...
I20260812 21:03:43.384537 16709 ts_manager.cc:194] Registered new tserver with Master: 0065827d12df4186936238d649da83db (127.11.254.131:45353)
I20260812 21:03:43.384887 16709 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.254.131@KRBTEST.COM'} at 127.11.254.131:54573
I20260812 21:03:43.389225 12282 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260812 21:03:43.402586 16783 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:03:43.401579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41428 (local address 127.11.254.190:45795)
0812 21:03:43.401663 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0812 21:03:43.401665 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:03:43.401744 (+ 79us) server_negotiation.cc:407] Connection header received
0812 21:03:43.401806 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:03:43.401807 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:03:43.401818 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:03:43.401869 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:03:43.402516 (+ 647us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41428: BlockingRecv error: recv got EOF from 127.0.0.1:41428 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 21:03:43.403952 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 16818
I20260812 21:03:43.411078 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 17085
I20260812 21:03:43.428699 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 17335
I20260812 21:03:43.445080 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 16649
2026-08-12T21:03:43Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (2007 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786568607354921-12282-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 21:03:43 dist-test-slave-vpvm krb5kdc[17582](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 21:03:43 dist-test-slave-vpvm krb5kdc[17582](info): set up 2 sockets
Aug 12 21:03:43 dist-test-slave-vpvm krb5kdc[17582](info): commencing operation
krb5kdc: starting...
W20260812 21:03:44.642683 12282 mini_kdc.cc:121] Time spent starting KDC: real 1.173s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786568607354921-12282-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786568607354921-12282-0/krb5kdc/test-user.keytab.
Aug 12 21:03:44 dist-test-slave-vpvm krb5kdc[17582](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568624, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T21:03:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T21:03:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.254.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.254.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.254.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.190.keytab.
Entry for principal kudu/127.11.254.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.190.keytab.
I20260812 21:03:44.769213 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786568607354921-12282-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786568607354921-12282-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786568607354921-12282-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.254.190:44201
--webserver_interface=127.11.254.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.190.keytab
--principal=kudu/127.11.254.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.254.148:42371
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.254.190:44201
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786568607354921-12282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786568607354921-12282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786568607354921-12282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:03:44.851831 17604 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:03:44.852036 17604 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:03:44.852064 17604 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:03:44.853204 17604 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 21:03:44.853248 17604 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:03:44.853266 17604 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:03:44.853282 17604 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 21:03:44.853298 17604 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 21:03:44.854823 17604 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.254.148:42371
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786568607354921-12282-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786568607354921-12282-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.254.190:44201
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.190.keytab
--principal=kudu/127.11.254.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.254.190:44201
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.254.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17604
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786568607354921-12282-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:03:44.855075 17604 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:03:44.855293 17604 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:03:44.860081 17612 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:03:44.860088 17611 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:03:44.860088 17616 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:03:44.860423 17604 server_base.cc:1034] running on GCE node
I20260812 21:03:44.860683 17604 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:03:44.860947 17604 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:03:44.862090 17604 hybrid_clock.cc:648] HybridClock initialized: now 1786568624862069 us; error 30 us; skew 500 ppm
Aug 12 21:03:44 dist-test-slave-vpvm krb5kdc[17582](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568624, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:03:44.867663 17604 init.cc:377] Logged in from keytab as kudu/127.11.254.190@KRBTEST.COM (short username kudu)
I20260812 21:03:44.868120 17604 webserver.cc:460] Webserver started at http://127.11.254.190:44485/ using document root <none> and password file <none>
I20260812 21:03:44.868346 17604 fs_manager.cc:359] Metadata directory not provided
I20260812 21:03:44.868396 17604 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:03:44.868525 17604 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:03:44.869598 17604 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786568607354921-12282-0/minicluster-data/master-0/data/instance:
uuid: "c5cfde80412f4e2bb0fa174fe6ab7de4"
format_stamp: "Formatted at 2026-08-12 21:03:44 on dist-test-slave-vpvm"
server_key: "ad828787186f64bc43b40e10b24b1e78"
server_key_iv: "d4799f9356a5d5d04a0c8a092a123b4b"
server_key_version: "encryptionkey@0"
I20260812 21:03:44.869939 17604 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786568607354921-12282-0/minicluster-data/master-0/wal/instance:
uuid: "c5cfde80412f4e2bb0fa174fe6ab7de4"
format_stamp: "Formatted at 2026-08-12 21:03:44 on dist-test-slave-vpvm"
server_key: "ad828787186f64bc43b40e10b24b1e78"
server_key_iv: "d4799f9356a5d5d04a0c8a092a123b4b"
server_key_version: "encryptionkey@0"
I20260812 21:03:44.871305 17604 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 21:03:44.872046 17626 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:03:44.872205 17604 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 21:03:44.872292 17604 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786568607354921-12282-0/minicluster-data/master-0/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786568607354921-12282-0/minicluster-data/master-0/wal
uuid: "c5cfde80412f4e2bb0fa174fe6ab7de4"
format_stamp: "Formatted at 2026-08-12 21:03:44 on dist-test-slave-vpvm"
server_key: "ad828787186f64bc43b40e10b24b1e78"
server_key_iv: "d4799f9356a5d5d04a0c8a092a123b4b"
server_key_version: "encryptionkey@0"
I20260812 21:03:44.872354 17604 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786568607354921-12282-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786568607354921-12282-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786568607354921-12282-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:03:44.882541 17604 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:03:44.884330 17604 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:03:44.884473 17604 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:03:44.888459 17604 rpc_server.cc:307] RPC server started. Bound to: 127.11.254.190:44201
I20260812 21:03:44.888527 17719 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.254.190:44201 every 8 connection(s)
I20260812 21:03:44.888970 17604 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
I20260812 21:03:44.889663 17722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:03:44.892030 17722 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c5cfde80412f4e2bb0fa174fe6ab7de4: Bootstrap starting.
I20260812 21:03:44.892616 17722 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c5cfde80412f4e2bb0fa174fe6ab7de4: Neither blocks nor log segments found. Creating new log.
I20260812 21:03:44.892900 17722 log.cc:824] T 00000000000000000000000000000000 P c5cfde80412f4e2bb0fa174fe6ab7de4: Log is configured to *not* fsync() on all Append() calls
I20260812 21:03:44.893572 17722 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c5cfde80412f4e2bb0fa174fe6ab7de4: No bootstrap required, opened a new log
I20260812 21:03:44.894861 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 17604
I20260812 21:03:44.894954 12282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786568607354921-12282-0/minicluster-data/master-0/wal/instance
I20260812 21:03:44.895126 12282 external_mini_cluster.cc:1428] Setting key 87a8adad32454e96699e243a98613452
I20260812 21:03:44.895447 17722 raft_consensus.cc:348] T 00000000000000000000000000000000 P c5cfde80412f4e2bb0fa174fe6ab7de4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5cfde80412f4e2bb0fa174fe6ab7de4" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 44201 } }
I20260812 21:03:44.895648 17722 raft_consensus.cc:374] T 00000000000000000000000000000000 P c5cfde80412f4e2bb0fa174fe6ab7de4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:03:44.895713 17722 raft_consensus.cc:729] T 00000000000000000000000000000000 P c5cfde80412f4e2bb0fa174fe6ab7de4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5cfde80412f4e2bb0fa174fe6ab7de4, State: Initialized, Role: FOLLOWER
I20260812 21:03:44.895838 17722 consensus_queue.cc:260] T 00000000000000000000000000000000 P c5cfde80412f4e2bb0fa174fe6ab7de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5cfde80412f4e2bb0fa174fe6ab7de4" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 44201 } }
I20260812 21:03:44.895905 17722 raft_consensus.cc:388] T 00000000000000000000000000000000 P c5cfde80412f4e2bb0fa174fe6ab7de4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 21:03:44.895936 17722 raft_consensus.cc:482] T 00000000000000000000000000000000 P c5cfde80412f4e2bb0fa174fe6ab7de4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 21:03:44.895980 17722 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c5cfde80412f4e2bb0fa174fe6ab7de4 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:03:44.896643 17722 raft_consensus.cc:504] T 00000000000000000000000000000000 P c5cfde80412f4e2bb0fa174fe6ab7de4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5cfde80412f4e2bb0fa174fe6ab7de4" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 44201 } }
I20260812 21:03:44.896756 17722 leader_election.cc:302] T 00000000000000000000000000000000 P c5cfde80412f4e2bb0fa174fe6ab7de4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c5cfde80412f4e2bb0fa174fe6ab7de4; no voters:
I20260812 21:03:44.896970 17722 leader_election.cc:288] T 00000000000000000000000000000000 P c5cfde80412f4e2bb0fa174fe6ab7de4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 21:03:44.897014 17732 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c5cfde80412f4e2bb0fa174fe6ab7de4 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:03:44.897156 17732 raft_consensus.cc:686] T 00000000000000000000000000000000 P c5cfde80412f4e2bb0fa174fe6ab7de4 [term 1 LEADER]: Becoming Leader. State: Replica: c5cfde80412f4e2bb0fa174fe6ab7de4, State: Running, Role: LEADER
I20260812 21:03:44.897269 17722 sys_catalog.cc:564] T 00000000000000000000000000000000 P c5cfde80412f4e2bb0fa174fe6ab7de4 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 21:03:44.897320 17732 consensus_queue.cc:237] T 00000000000000000000000000000000 P c5cfde80412f4e2bb0fa174fe6ab7de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5cfde80412f4e2bb0fa174fe6ab7de4" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 44201 } }
I20260812 21:03:44.897656 17732 sys_catalog.cc:455] T 00000000000000000000000000000000 P c5cfde80412f4e2bb0fa174fe6ab7de4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c5cfde80412f4e2bb0fa174fe6ab7de4. Latest consensus state: current_term: 1 leader_uuid: "c5cfde80412f4e2bb0fa174fe6ab7de4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5cfde80412f4e2bb0fa174fe6ab7de4" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 44201 } } }
I20260812 21:03:44.897742 17732 sys_catalog.cc:458] T 00000000000000000000000000000000 P c5cfde80412f4e2bb0fa174fe6ab7de4 [sys.catalog]: This master's current role is: LEADER
I20260812 21:03:44.897667 17734 sys_catalog.cc:455] T 00000000000000000000000000000000 P c5cfde80412f4e2bb0fa174fe6ab7de4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c5cfde80412f4e2bb0fa174fe6ab7de4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5cfde80412f4e2bb0fa174fe6ab7de4" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 44201 } } }
I20260812 21:03:44.897997 17734 sys_catalog.cc:458] T 00000000000000000000000000000000 P c5cfde80412f4e2bb0fa174fe6ab7de4 [sys.catalog]: This master's current role is: LEADER
I20260812 21:03:44.898145 17743 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 21:03:44.899288 17743 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 21:03:44.901307 17743 catalog_manager.cc:1347] Generated new cluster ID: c55f1725c7034320ab6d8e7cb5ed5c0f
I20260812 21:03:44.901356 17743 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 21:03:44 dist-test-slave-vpvm krb5kdc[17582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568624, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.254.190@KRBTEST.COM
I20260812 21:03:44.906347 17729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:44.895638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57016 (local address 127.11.254.190:44201)
0812 21:03:44.896414 (+ 776us) server_negotiation.cc:206] Beginning negotiation
0812 21:03:44.896424 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 21:03:44.896459 (+ 35us) server_negotiation.cc:407] Connection header received
0812 21:03:44.896577 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:03:44.896585 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:03:44.896644 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:03:44.896791 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:03:44.897590 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:44.899052 (+ 1462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:44.900118 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:44.900750 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:44.900876 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:44.903234 (+ 2358us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:03:44.903239 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:03:44.903239 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:03:44.903251 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:03:44.904815 (+ 1564us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:44.905163 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:44.905166 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:44.905166 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:44.905266 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:44.905446 (+ 180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:44.905447 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:44.905447 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:44.905702 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:03:44.905849 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0812 21:03:44.905974 (+ 125us) server_negotiation.cc:299] Negotiation successful
0812 21:03:44.906147 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":702,"thread_start_us":653,"threads_started":1}
I20260812 21:03:44.907189 12282 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 21:03:44.922245 17729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:44.918778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57030 (local address 127.11.254.190:44201)
0812 21:03:44.918835 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0812 21:03:44.918837 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:03:44.919029 (+ 192us) server_negotiation.cc:407] Connection header received
0812 21:03:44.919100 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:03:44.919101 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:03:44.919114 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:03:44.919164 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:03:44.919573 (+ 409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:44.920094 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:44.920807 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:44.921033 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:44.921091 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:44.921735 (+ 644us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:03:44.921737 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:03:44.921737 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 21:03:44.921938 (+ 201us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:03:44.922000 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 21:03:44.922008 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:03:44.922061 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0812 21:03:44.922134 (+ 73us) server_negotiation.cc:299] Negotiation successful
0812 21:03:44.922159 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260812 21:03:44.923874 17743 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 21:03:44.924382 17743 catalog_manager.cc:1504] Loading token signing keys...
I20260812 21:03:44.929296 17743 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c5cfde80412f4e2bb0fa174fe6ab7de4: Generated new TSK 0
I20260812 21:03:44.929525 17743 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 21:03:44.939137 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 17604
2026-08-12T21:03:44Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1494 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568607354921-12282-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 21:03:44 dist-test-slave-vpvm krb5kdc[17778](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 21:03:45 dist-test-slave-vpvm krb5kdc[17778](info): set up 2 sockets
Aug 12 21:03:45 dist-test-slave-vpvm krb5kdc[17778](info): commencing operation
krb5kdc: starting...
W20260812 21:03:46.124130 12282 mini_kdc.cc:121] Time spent starting KDC: real 1.159s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568607354921-12282-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568607354921-12282-0/krb5kdc/test-user.keytab.
Aug 12 21:03:46 dist-test-slave-vpvm krb5kdc[17778](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568626, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T21:03:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T21:03:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.254.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.254.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.254.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.190.keytab.
Entry for principal kudu/127.11.254.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.190.keytab.
I20260812 21:03:46.253490 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568607354921-12282-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568607354921-12282-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568607354921-12282-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.254.190:39885
--webserver_interface=127.11.254.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.190.keytab
--principal=kudu/127.11.254.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.254.148:46687
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.254.190:39885
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568607354921-12282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568607354921-12282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568607354921-12282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:03:46.338940 17802 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:03:46.339182 17802 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:03:46.339206 17802 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:03:46.340453 17802 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 21:03:46.340509 17802 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:03:46.340523 17802 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:03:46.340534 17802 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 21:03:46.340544 17802 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 21:03:46.342118 17802 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.254.148:46687
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568607354921-12282-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568607354921-12282-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.254.190:39885
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.190.keytab
--principal=kudu/127.11.254.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.254.190:39885
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.254.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17802
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568607354921-12282-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:03:46.342365 17802 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:03:46.342600 17802 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:03:46.347592 17816 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:03:46.347591 17809 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:03:46.347625 17812 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:03:46.347874 17802 server_base.cc:1034] running on GCE node
I20260812 21:03:46.348160 17802 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:03:46.348521 17802 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:03:46.349676 17802 hybrid_clock.cc:648] HybridClock initialized: now 1786568626349658 us; error 37 us; skew 500 ppm
Aug 12 21:03:46 dist-test-slave-vpvm krb5kdc[17778](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568626, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:03:46.352264 17802 init.cc:377] Logged in from keytab as kudu/127.11.254.190@KRBTEST.COM (short username kudu)
I20260812 21:03:46.352684 17802 webserver.cc:460] Webserver started at http://127.11.254.190:35965/ using document root <none> and password file <none>
I20260812 21:03:46.352865 17802 fs_manager.cc:359] Metadata directory not provided
I20260812 21:03:46.352906 17802 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:03:46.352999 17802 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:03:46.354106 17802 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568607354921-12282-0/minicluster-data/master-0/data/instance:
uuid: "742d8d89bb61406ebeaa162c0e1b85f4"
format_stamp: "Formatted at 2026-08-12 21:03:46 on dist-test-slave-vpvm"
server_key: "4a7c103ab4d03526946e313b0e50f5ff"
server_key_iv: "189935cb97ab4980a09551c4564d56e6"
server_key_version: "encryptionkey@0"
I20260812 21:03:46.354466 17802 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568607354921-12282-0/minicluster-data/master-0/wal/instance:
uuid: "742d8d89bb61406ebeaa162c0e1b85f4"
format_stamp: "Formatted at 2026-08-12 21:03:46 on dist-test-slave-vpvm"
server_key: "4a7c103ab4d03526946e313b0e50f5ff"
server_key_iv: "189935cb97ab4980a09551c4564d56e6"
server_key_version: "encryptionkey@0"
I20260812 21:03:46.355783 17802 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 21:03:46.356556 17824 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:03:46.356739 17802 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 21:03:46.356827 17802 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568607354921-12282-0/minicluster-data/master-0/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568607354921-12282-0/minicluster-data/master-0/wal
uuid: "742d8d89bb61406ebeaa162c0e1b85f4"
format_stamp: "Formatted at 2026-08-12 21:03:46 on dist-test-slave-vpvm"
server_key: "4a7c103ab4d03526946e313b0e50f5ff"
server_key_iv: "189935cb97ab4980a09551c4564d56e6"
server_key_version: "encryptionkey@0"
I20260812 21:03:46.356885 17802 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568607354921-12282-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568607354921-12282-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568607354921-12282-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:03:46.370196 17802 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:03:46.372943 17802 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:03:46.373121 17802 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:03:46.378489 17802 rpc_server.cc:307] RPC server started. Bound to: 127.11.254.190:39885
I20260812 21:03:46.378538 17928 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.254.190:39885 every 8 connection(s)
I20260812 21:03:46.378985 17802 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
I20260812 21:03:46.381397 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 17802
I20260812 21:03:46.381494 12282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568607354921-12282-0/minicluster-data/master-0/wal/instance
I20260812 21:03:46.381668 12282 external_mini_cluster.cc:1428] Setting key 60563a109efa1f0cbe441b11247adfd5
I20260812 21:03:46.382572 17929 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:03:46.385998 17929 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 742d8d89bb61406ebeaa162c0e1b85f4: Bootstrap starting.
I20260812 21:03:46.386603 17929 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 742d8d89bb61406ebeaa162c0e1b85f4: Neither blocks nor log segments found. Creating new log.
I20260812 21:03:46.386914 17929 log.cc:824] T 00000000000000000000000000000000 P 742d8d89bb61406ebeaa162c0e1b85f4: Log is configured to *not* fsync() on all Append() calls
I20260812 21:03:46.387722 17929 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 742d8d89bb61406ebeaa162c0e1b85f4: No bootstrap required, opened a new log
Aug 12 21:03:46 dist-test-slave-vpvm krb5kdc[17778](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568626, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.254.190@KRBTEST.COM
I20260812 21:03:46.389734 17929 raft_consensus.cc:348] T 00000000000000000000000000000000 P 742d8d89bb61406ebeaa162c0e1b85f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "742d8d89bb61406ebeaa162c0e1b85f4" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 39885 } }
I20260812 21:03:46.389925 17929 raft_consensus.cc:374] T 00000000000000000000000000000000 P 742d8d89bb61406ebeaa162c0e1b85f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:03:46.389999 17929 raft_consensus.cc:729] T 00000000000000000000000000000000 P 742d8d89bb61406ebeaa162c0e1b85f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 742d8d89bb61406ebeaa162c0e1b85f4, State: Initialized, Role: FOLLOWER
I20260812 21:03:46.390141 17929 consensus_queue.cc:260] T 00000000000000000000000000000000 P 742d8d89bb61406ebeaa162c0e1b85f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "742d8d89bb61406ebeaa162c0e1b85f4" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 39885 } }
I20260812 21:03:46.390244 17929 raft_consensus.cc:388] T 00000000000000000000000000000000 P 742d8d89bb61406ebeaa162c0e1b85f4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 21:03:46.390286 17929 raft_consensus.cc:482] T 00000000000000000000000000000000 P 742d8d89bb61406ebeaa162c0e1b85f4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 21:03:46.390337 17929 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 742d8d89bb61406ebeaa162c0e1b85f4 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:03:46.391076 17929 raft_consensus.cc:504] T 00000000000000000000000000000000 P 742d8d89bb61406ebeaa162c0e1b85f4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "742d8d89bb61406ebeaa162c0e1b85f4" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 39885 } }
I20260812 21:03:46.391193 17929 leader_election.cc:302] T 00000000000000000000000000000000 P 742d8d89bb61406ebeaa162c0e1b85f4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 742d8d89bb61406ebeaa162c0e1b85f4; no voters:
I20260812 21:03:46.391366 17929 leader_election.cc:288] T 00000000000000000000000000000000 P 742d8d89bb61406ebeaa162c0e1b85f4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 21:03:46.391408 17937 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 742d8d89bb61406ebeaa162c0e1b85f4 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:03:46.391525 17937 raft_consensus.cc:686] T 00000000000000000000000000000000 P 742d8d89bb61406ebeaa162c0e1b85f4 [term 1 LEADER]: Becoming Leader. State: Replica: 742d8d89bb61406ebeaa162c0e1b85f4, State: Running, Role: LEADER
I20260812 21:03:46.391606 17937 consensus_queue.cc:237] T 00000000000000000000000000000000 P 742d8d89bb61406ebeaa162c0e1b85f4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "742d8d89bb61406ebeaa162c0e1b85f4" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 39885 } }
I20260812 21:03:46.391640 17929 sys_catalog.cc:564] T 00000000000000000000000000000000 P 742d8d89bb61406ebeaa162c0e1b85f4 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 21:03:46.391938 17937 sys_catalog.cc:455] T 00000000000000000000000000000000 P 742d8d89bb61406ebeaa162c0e1b85f4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 742d8d89bb61406ebeaa162c0e1b85f4. Latest consensus state: current_term: 1 leader_uuid: "742d8d89bb61406ebeaa162c0e1b85f4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "742d8d89bb61406ebeaa162c0e1b85f4" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 39885 } } }
I20260812 21:03:46.392017 17937 sys_catalog.cc:458] T 00000000000000000000000000000000 P 742d8d89bb61406ebeaa162c0e1b85f4 [sys.catalog]: This master's current role is: LEADER
I20260812 21:03:46.391947 17939 sys_catalog.cc:455] T 00000000000000000000000000000000 P 742d8d89bb61406ebeaa162c0e1b85f4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "742d8d89bb61406ebeaa162c0e1b85f4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "742d8d89bb61406ebeaa162c0e1b85f4" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 39885 } } }
I20260812 21:03:46.392170 17939 sys_catalog.cc:458] T 00000000000000000000000000000000 P 742d8d89bb61406ebeaa162c0e1b85f4 [sys.catalog]: This master's current role is: LEADER
I20260812 21:03:46.392823 17946 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 21:03:46.393329 17946 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 21:03:46.394135 17935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:46.382474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39698 (local address 127.11.254.190:39885)
0812 21:03:46.382650 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0812 21:03:46.382660 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 21:03:46.382692 (+ 32us) server_negotiation.cc:407] Connection header received
0812 21:03:46.383274 (+ 582us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:03:46.383284 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:03:46.383354 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:03:46.383504 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:03:46.384381 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:46.385745 (+ 1364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:46.386678 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:46.387252 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:46.387352 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:46.390227 (+ 2875us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:03:46.390231 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:03:46.390232 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:03:46.390246 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:03:46.392473 (+ 2227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:46.392907 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:46.392910 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:46.392910 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:46.393023 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:46.393233 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:46.393234 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:46.393235 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:46.393499 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:03:46.393642 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0812 21:03:46.393760 (+ 118us) server_negotiation.cc:299] Negotiation successful
0812 21:03:46.393906 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":71,"threads_started":1}
I20260812 21:03:46.394901 17946 catalog_manager.cc:1347] Generated new cluster ID: a9097ea41d7847db9c5dc97a0654aece
I20260812 21:03:46.394951 17946 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 21:03:46.408607 17946 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 21:03:46.409199 17946 catalog_manager.cc:1504] Loading token signing keys...
I20260812 21:03:46.416720 17946 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 742d8d89bb61406ebeaa162c0e1b85f4: Generated new TSK 0
I20260812 21:03:46.416939 17946 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.254.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.254.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.254.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.129.keytab.
Entry for principal kudu/127.11.254.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.129.keytab.
I20260812 21:03:46.448696 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568607354921-12282-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568607354921-12282-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568607354921-12282-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568607354921-12282-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.254.129:0
--local_ip_for_outbound_sockets=127.11.254.129
--webserver_interface=127.11.254.129
--webserver_port=0
--tserver_master_addrs=127.11.254.190:39885
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.129.keytab
--principal=kudu/127.11.254.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.254.148:46687
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568607354921-12282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568607354921-12282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568607354921-12282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:03:46.543313 17974 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:03:46.543530 17974 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:03:46.543552 17974 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:03:46.544682 17974 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:03:46.544724 17974 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:03:46.544749 17974 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.254.129
I20260812 21:03:46.546245 17974 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.254.148:46687
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568607354921-12282-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568607354921-12282-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.129.keytab
--principal=kudu/127.11.254.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.254.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568607354921-12282-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.254.129
--webserver_port=0
--tserver_master_addrs=127.11.254.190:39885
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17974
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.254.129
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568607354921-12282-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:03:46.546491 17974 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:03:46.546701 17974 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:03:46.551611 17981 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:03:46.551630 17983 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:03:46.551774 17974 server_base.cc:1034] running on GCE node
W20260812 21:03:46.551599 17980 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:03:46.552095 17974 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:03:46.552336 17974 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:03:46.553524 17974 hybrid_clock.cc:648] HybridClock initialized: now 1786568626553504 us; error 36 us; skew 500 ppm
Aug 12 21:03:46 dist-test-slave-vpvm krb5kdc[17778](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568626, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:03:46.556080 17974 init.cc:377] Logged in from keytab as kudu/127.11.254.129@KRBTEST.COM (short username kudu)
I20260812 21:03:46.556499 17974 webserver.cc:460] Webserver started at http://127.11.254.129:33661/ using document root <none> and password file <none>
I20260812 21:03:46.556710 17974 fs_manager.cc:359] Metadata directory not provided
I20260812 21:03:46.556754 17974 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:03:46.556862 17974 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:03:46.558003 17974 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568607354921-12282-0/minicluster-data/ts-0/data/instance:
uuid: "726ae4bd1d6b48aca6589efc580439bc"
format_stamp: "Formatted at 2026-08-12 21:03:46 on dist-test-slave-vpvm"
server_key: "5a0ce4377960053dba50c5c149942655"
server_key_iv: "ecf36bad77d36b5c6c3207e99d231e9a"
server_key_version: "encryptionkey@0"
I20260812 21:03:46.558367 17974 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568607354921-12282-0/minicluster-data/ts-0/wal/instance:
uuid: "726ae4bd1d6b48aca6589efc580439bc"
format_stamp: "Formatted at 2026-08-12 21:03:46 on dist-test-slave-vpvm"
server_key: "5a0ce4377960053dba50c5c149942655"
server_key_iv: "ecf36bad77d36b5c6c3207e99d231e9a"
server_key_version: "encryptionkey@0"
I20260812 21:03:46.559738 17974 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 21:03:46.560474 17998 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:03:46.560650 17974 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 21:03:46.560742 17974 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568607354921-12282-0/minicluster-data/ts-0/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568607354921-12282-0/minicluster-data/ts-0/wal
uuid: "726ae4bd1d6b48aca6589efc580439bc"
format_stamp: "Formatted at 2026-08-12 21:03:46 on dist-test-slave-vpvm"
server_key: "5a0ce4377960053dba50c5c149942655"
server_key_iv: "ecf36bad77d36b5c6c3207e99d231e9a"
server_key_version: "encryptionkey@0"
I20260812 21:03:46.560811 17974 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568607354921-12282-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568607354921-12282-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568607354921-12282-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:03:46.587661 17974 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:03:46.589485 17974 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:03:46.589609 17974 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:03:46.589813 17974 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:03:46.590149 17974 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:03:46.590189 17974 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:03:46.590211 17974 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:03:46.590233 17974 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:03:46.596012 17974 rpc_server.cc:307] RPC server started. Bound to: 127.11.254.129:33275
I20260812 21:03:46.596071 18204 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.254.129:33275 every 8 connection(s)
I20260812 21:03:46.596371 17974 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568607354921-12282-0/minicluster-data/ts-0/data/info.pb
Aug 12 21:03:46 dist-test-slave-vpvm krb5kdc[17778](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568626, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.129@KRBTEST.COM for kudu/127.11.254.190@KRBTEST.COM
I20260812 21:03:46.605357 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 17974
I20260812 21:03:46.605453 12282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568607354921-12282-0/minicluster-data/ts-0/wal/instance
I20260812 21:03:46.605636 12282 external_mini_cluster.cc:1428] Setting key 7026ce1d534a2f17907aefeb63be0c7f
I20260812 21:03:46.606249 17935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:46.597074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.129:57071 (local address 127.11.254.190:39885)
0812 21:03:46.597141 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0812 21:03:46.597143 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:03:46.597359 (+ 216us) server_negotiation.cc:407] Connection header received
0812 21:03:46.597890 (+ 531us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:03:46.597891 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:03:46.597903 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:03:46.597983 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:03:46.598932 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:46.599510 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:46.600509 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:46.600730 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:46.600787 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:46.603040 (+ 2253us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:03:46.603042 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:03:46.603043 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:03:46.603051 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:03:46.604414 (+ 1363us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:46.604931 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:46.604933 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:46.604933 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:46.605004 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:46.605317 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:46.605319 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:46.605319 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:46.605553 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:03:46.605633 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0812 21:03:46.606032 (+ 399us) server_negotiation.cc:299] Negotiation successful
0812 21:03:46.606151 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260812 21:03:46.606408 18210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:46.597056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:39885 (local address 127.11.254.129:57071)
0812 21:03:46.597226 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0812 21:03:46.597267 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 21:03:46.597785 (+ 518us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:03:46.598051 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:03:46.598055 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:03:46.598632 (+ 577us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:03:46.598833 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:46.598838 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:46.599616 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:46.599617 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:46.600422 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:46.600424 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:46.600807 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:46.600808 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:46.600932 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:46.601410 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:03:46.601419 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:03:46.602947 (+ 1528us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:03:46.604510 (+ 1563us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:03:46.604520 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:03:46.604523 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:03:46.604789 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:03:46.605067 (+ 278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:03:46.605068 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:03:46.605069 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:03:46.605217 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:03:46.605650 (+ 433us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:03:46.605659 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:03:46.605837 (+ 178us) client_negotiation.cc:769] Sending connection context
0812 21:03:46.605991 (+ 154us) client_negotiation.cc:240] Negotiation successful
0812 21:03:46.606177 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":88,"threads_started":1}
I20260812 21:03:46.606938 18206 heartbeater.cc:347] Connected to a master server at 127.11.254.190:39885
I20260812 21:03:46.607056 18206 heartbeater.cc:464] Registering TS with master...
I20260812 21:03:46.607254 18206 heartbeater.cc:511] Master 127.11.254.190:39885 requested a full tablet report, sending...
I20260812 21:03:46.607829 17858 ts_manager.cc:194] Registered new tserver with Master: 726ae4bd1d6b48aca6589efc580439bc (127.11.254.129:33275)
I20260812 21:03:46.608546 17858 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.254.129@KRBTEST.COM'} at 127.11.254.129:57071
WARNING: no policy specified for kudu/127.11.254.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.254.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.254.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.130.keytab.
Entry for principal kudu/127.11.254.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.130.keytab.
I20260812 21:03:46.632936 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568607354921-12282-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568607354921-12282-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568607354921-12282-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568607354921-12282-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.254.130:0
--local_ip_for_outbound_sockets=127.11.254.130
--webserver_interface=127.11.254.130
--webserver_port=0
--tserver_master_addrs=127.11.254.190:39885
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.130.keytab
--principal=kudu/127.11.254.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.254.148:46687
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568607354921-12282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568607354921-12282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568607354921-12282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:03:46.717283 18220 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:03:46.717509 18220 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:03:46.717533 18220 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:03:46.718724 18220 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:03:46.718768 18220 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:03:46.718794 18220 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.254.130
I20260812 21:03:46.720292 18220 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.254.148:46687
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568607354921-12282-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568607354921-12282-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.130.keytab
--principal=kudu/127.11.254.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.254.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568607354921-12282-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.254.130
--webserver_port=0
--tserver_master_addrs=127.11.254.190:39885
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18220
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.254.130
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568607354921-12282-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:03:46.720530 18220 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:03:46.720772 18220 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:03:46.725795 18239 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:03:46.725828 18232 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:03:46.725795 18231 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:03:46.726071 18220 server_base.cc:1034] running on GCE node
I20260812 21:03:46.726279 18220 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:03:46.726511 18220 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:03:46.727677 18220 hybrid_clock.cc:648] HybridClock initialized: now 1786568626727660 us; error 28 us; skew 500 ppm
Aug 12 21:03:46 dist-test-slave-vpvm krb5kdc[17778](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568626, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:03:46.730309 18220 init.cc:377] Logged in from keytab as kudu/127.11.254.130@KRBTEST.COM (short username kudu)
I20260812 21:03:46.730798 18220 webserver.cc:460] Webserver started at http://127.11.254.130:34305/ using document root <none> and password file <none>
I20260812 21:03:46.731029 18220 fs_manager.cc:359] Metadata directory not provided
I20260812 21:03:46.731082 18220 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:03:46.731207 18220 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:03:46.732259 18220 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568607354921-12282-0/minicluster-data/ts-1/data/instance:
uuid: "ad743e7bb342456ca18f961f87ca9437"
format_stamp: "Formatted at 2026-08-12 21:03:46 on dist-test-slave-vpvm"
server_key: "270b7c4e9062b945b514326e06f4ac90"
server_key_iv: "372cfd80e6557d7cd29b1a5640d679f3"
server_key_version: "encryptionkey@0"
I20260812 21:03:46.732605 18220 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568607354921-12282-0/minicluster-data/ts-1/wal/instance:
uuid: "ad743e7bb342456ca18f961f87ca9437"
format_stamp: "Formatted at 2026-08-12 21:03:46 on dist-test-slave-vpvm"
server_key: "270b7c4e9062b945b514326e06f4ac90"
server_key_iv: "372cfd80e6557d7cd29b1a5640d679f3"
server_key_version: "encryptionkey@0"
I20260812 21:03:46.733891 18220 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260812 21:03:46.734668 18256 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:03:46.734817 18220 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 21:03:46.734900 18220 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568607354921-12282-0/minicluster-data/ts-1/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568607354921-12282-0/minicluster-data/ts-1/wal
uuid: "ad743e7bb342456ca18f961f87ca9437"
format_stamp: "Formatted at 2026-08-12 21:03:46 on dist-test-slave-vpvm"
server_key: "270b7c4e9062b945b514326e06f4ac90"
server_key_iv: "372cfd80e6557d7cd29b1a5640d679f3"
server_key_version: "encryptionkey@0"
I20260812 21:03:46.734961 18220 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568607354921-12282-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568607354921-12282-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568607354921-12282-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:03:46.752524 18220 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:03:46.754483 18220 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:03:46.754629 18220 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:03:46.754869 18220 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:03:46.755180 18220 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:03:46.755215 18220 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:03:46.755245 18220 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:03:46.755267 18220 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:03:46.760993 18220 rpc_server.cc:307] RPC server started. Bound to: 127.11.254.130:37843
I20260812 21:03:46.761049 18440 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.254.130:37843 every 8 connection(s)
I20260812 21:03:46.761356 18220 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568607354921-12282-0/minicluster-data/ts-1/data/info.pb
Aug 12 21:03:46 dist-test-slave-vpvm krb5kdc[17778](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568626, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.130@KRBTEST.COM for kudu/127.11.254.190@KRBTEST.COM
I20260812 21:03:46.768947 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 18220
I20260812 21:03:46.769058 12282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568607354921-12282-0/minicluster-data/ts-1/wal/instance
I20260812 21:03:46.769260 12282 external_mini_cluster.cc:1428] Setting key 0d215664ba48936f9f3e18442cde86ba
I20260812 21:03:46.770588 17935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:46.762009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.130:37319 (local address 127.11.254.190:39885)
0812 21:03:46.762070 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0812 21:03:46.762072 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:03:46.762264 (+ 192us) server_negotiation.cc:407] Connection header received
0812 21:03:46.762800 (+ 536us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:03:46.762801 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:03:46.762814 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:03:46.762866 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:03:46.763894 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:46.764436 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:46.765431 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:46.765623 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:46.765678 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:46.767658 (+ 1980us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:03:46.767661 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:03:46.767661 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:03:46.767669 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:03:46.769016 (+ 1347us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:46.769533 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:46.769535 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:46.769535 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:46.769605 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:46.769828 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:46.769829 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:46.769829 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:46.769997 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:03:46.770060 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0812 21:03:46.770444 (+ 384us) server_negotiation.cc:299] Negotiation successful
0812 21:03:46.770522 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260812 21:03:46.770753 18443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:46.761990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:39885 (local address 127.11.254.130:37319)
0812 21:03:46.762150 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0812 21:03:46.762186 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0812 21:03:46.762695 (+ 509us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:03:46.762961 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:03:46.762965 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:03:46.763597 (+ 632us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:03:46.763796 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:46.763801 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:46.764531 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:46.764532 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:46.765336 (+ 804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:46.765339 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:46.765691 (+ 352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:46.765692 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:46.765815 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:46.766280 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:03:46.766289 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:03:46.767571 (+ 1282us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:03:46.769115 (+ 1544us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:03:46.769122 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:03:46.769125 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:03:46.769423 (+ 298us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:03:46.769667 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:03:46.769668 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:03:46.769668 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:03:46.769774 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:03:46.770071 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:03:46.770077 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:03:46.770244 (+ 167us) client_negotiation.cc:769] Sending connection context
0812 21:03:46.770357 (+ 113us) client_negotiation.cc:240] Negotiation successful
0812 21:03:46.770527 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":82,"threads_started":1}
I20260812 21:03:46.771234 18441 heartbeater.cc:347] Connected to a master server at 127.11.254.190:39885
I20260812 21:03:46.771328 18441 heartbeater.cc:464] Registering TS with master...
I20260812 21:03:46.771509 18441 heartbeater.cc:511] Master 127.11.254.190:39885 requested a full tablet report, sending...
I20260812 21:03:46.771886 17858 ts_manager.cc:194] Registered new tserver with Master: ad743e7bb342456ca18f961f87ca9437 (127.11.254.130:37843)
I20260812 21:03:46.772253 17858 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.254.130@KRBTEST.COM'} at 127.11.254.130:37319
WARNING: no policy specified for kudu/127.11.254.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.254.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.254.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.131.keytab.
Entry for principal kudu/127.11.254.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.131.keytab.
I20260812 21:03:46.796582 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568607354921-12282-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568607354921-12282-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568607354921-12282-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568607354921-12282-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.254.131:0
--local_ip_for_outbound_sockets=127.11.254.131
--webserver_interface=127.11.254.131
--webserver_port=0
--tserver_master_addrs=127.11.254.190:39885
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.131.keytab
--principal=kudu/127.11.254.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.254.148:46687
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568607354921-12282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568607354921-12282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568607354921-12282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:03:46.880355 18447 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:03:46.880574 18447 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:03:46.880616 18447 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:03:46.881817 18447 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:03:46.881868 18447 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:03:46.881906 18447 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.254.131
I20260812 21:03:46.883461 18447 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.254.148:46687
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568607354921-12282-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568607354921-12282-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.131.keytab
--principal=kudu/127.11.254.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.254.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568607354921-12282-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.254.131
--webserver_port=0
--tserver_master_addrs=127.11.254.190:39885
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18447
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.254.131
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568607354921-12282-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:03:46.883734 18447 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:03:46.883955 18447 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:03:46.889096 18455 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:03:46.889072 18453 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:03:46.889057 18452 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:03:46.889221 18447 server_base.cc:1034] running on GCE node
I20260812 21:03:46.889441 18447 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:03:46.889679 18447 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:03:46.890820 18447 hybrid_clock.cc:648] HybridClock initialized: now 1786568626890803 us; error 29 us; skew 500 ppm
Aug 12 21:03:46 dist-test-slave-vpvm krb5kdc[17778](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568626, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:03:46.893504 18447 init.cc:377] Logged in from keytab as kudu/127.11.254.131@KRBTEST.COM (short username kudu)
I20260812 21:03:46.893970 18447 webserver.cc:460] Webserver started at http://127.11.254.131:42043/ using document root <none> and password file <none>
I20260812 21:03:46.894230 18447 fs_manager.cc:359] Metadata directory not provided
I20260812 21:03:46.894289 18447 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:03:46.894397 18447 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:03:46.895413 18447 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568607354921-12282-0/minicluster-data/ts-2/data/instance:
uuid: "ea6d6ce4a8af47109d8244c4086114c3"
format_stamp: "Formatted at 2026-08-12 21:03:46 on dist-test-slave-vpvm"
server_key: "4993439fcf7da42634ddf7ebcf498c88"
server_key_iv: "89df0b82ba4930600a8eed154cde3a61"
server_key_version: "encryptionkey@0"
I20260812 21:03:46.895747 18447 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568607354921-12282-0/minicluster-data/ts-2/wal/instance:
uuid: "ea6d6ce4a8af47109d8244c4086114c3"
format_stamp: "Formatted at 2026-08-12 21:03:46 on dist-test-slave-vpvm"
server_key: "4993439fcf7da42634ddf7ebcf498c88"
server_key_iv: "89df0b82ba4930600a8eed154cde3a61"
server_key_version: "encryptionkey@0"
I20260812 21:03:46.896997 18447 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 21:03:46.897643 18465 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:03:46.897792 18447 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 21:03:46.897876 18447 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568607354921-12282-0/minicluster-data/ts-2/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568607354921-12282-0/minicluster-data/ts-2/wal
uuid: "ea6d6ce4a8af47109d8244c4086114c3"
format_stamp: "Formatted at 2026-08-12 21:03:46 on dist-test-slave-vpvm"
server_key: "4993439fcf7da42634ddf7ebcf498c88"
server_key_iv: "89df0b82ba4930600a8eed154cde3a61"
server_key_version: "encryptionkey@0"
I20260812 21:03:46.897936 18447 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568607354921-12282-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568607354921-12282-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568607354921-12282-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:03:46.905794 18447 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:03:46.907614 18447 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:03:46.907743 18447 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:03:46.907956 18447 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:03:46.908249 18447 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:03:46.908282 18447 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:03:46.908309 18447 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:03:46.908334 18447 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:03:46.914160 18447 rpc_server.cc:307] RPC server started. Bound to: 127.11.254.131:34761
I20260812 21:03:46.914220 18670 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.254.131:34761 every 8 connection(s)
I20260812 21:03:46.914538 18447 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568607354921-12282-0/minicluster-data/ts-2/data/info.pb
Aug 12 21:03:46 dist-test-slave-vpvm krb5kdc[17778](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568626, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.131@KRBTEST.COM for kudu/127.11.254.190@KRBTEST.COM
I20260812 21:03:46.922101 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 18447
I20260812 21:03:46.922199 12282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786568607354921-12282-0/minicluster-data/ts-2/wal/instance
I20260812 21:03:46.922394 12282 external_mini_cluster.cc:1428] Setting key 63b969b5e5578e0c1ef7ddc1e563a6a2
I20260812 21:03:46.923962 17935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:46.915282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.131:47287 (local address 127.11.254.190:39885)
0812 21:03:46.915344 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0812 21:03:46.915346 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:03:46.915538 (+ 192us) server_negotiation.cc:407] Connection header received
0812 21:03:46.916076 (+ 538us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:03:46.916078 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:03:46.916092 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:03:46.916145 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:03:46.917265 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:46.917773 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:46.918812 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:46.919007 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:46.919064 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:46.921184 (+ 2120us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:03:46.921185 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:03:46.921186 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:03:46.921193 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:03:46.922456 (+ 1263us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:46.922881 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:46.922882 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:46.922882 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:46.922945 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:46.923280 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:46.923281 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:46.923281 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:46.923469 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:03:46.923540 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0812 21:03:46.923816 (+ 276us) server_negotiation.cc:299] Negotiation successful
0812 21:03:46.923895 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260812 21:03:46.924069 18674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:46.915217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:39885 (local address 127.11.254.131:47287)
0812 21:03:46.915393 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0812 21:03:46.915441 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 21:03:46.915968 (+ 527us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:03:46.916226 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:03:46.916231 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:03:46.916942 (+ 711us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:03:46.917167 (+ 225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:46.917173 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:46.917858 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:46.917859 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:46.918725 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:46.918728 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:46.919077 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:46.919078 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:46.919200 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:46.919714 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:03:46.919723 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:03:46.921094 (+ 1371us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:03:46.922538 (+ 1444us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:03:46.922544 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:03:46.922546 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:03:46.922808 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:03:46.923049 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:03:46.923050 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:03:46.923051 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:03:46.923219 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:03:46.923528 (+ 309us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:03:46.923534 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:03:46.923658 (+ 124us) client_negotiation.cc:769] Sending connection context
0812 21:03:46.923754 (+ 96us) client_negotiation.cc:240] Negotiation successful
0812 21:03:46.923885 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":70,"threads_started":1}
I20260812 21:03:46.924453 18672 heartbeater.cc:347] Connected to a master server at 127.11.254.190:39885
I20260812 21:03:46.924541 18672 heartbeater.cc:464] Registering TS with master...
I20260812 21:03:46.924722 18672 heartbeater.cc:511] Master 127.11.254.190:39885 requested a full tablet report, sending...
I20260812 21:03:46.925058 17858 ts_manager.cc:194] Registered new tserver with Master: ea6d6ce4a8af47109d8244c4086114c3 (127.11.254.131:34761)
I20260812 21:03:46.925431 17858 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.254.131@KRBTEST.COM'} at 127.11.254.131:47287
I20260812 21:03:46.934803 12282 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 12 21:03:46 dist-test-slave-vpvm krb5kdc[17778](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568626, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 12 21:03:46 dist-test-slave-vpvm krb5kdc[17778](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568626, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.11.254.190@KRBTEST.COM
I20260812 21:03:46.958109 17935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:46.951406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39714 (local address 127.11.254.190:39885)
0812 21:03:46.951460 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0812 21:03:46.951462 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:03:46.951588 (+ 126us) server_negotiation.cc:407] Connection header received
0812 21:03:46.951621 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:03:46.951622 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:03:46.951632 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:03:46.951689 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:03:46.952233 (+ 544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:46.952748 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:46.953506 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:46.953701 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:46.953745 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:46.955987 (+ 2242us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:03:46.955989 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:03:46.955989 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:03:46.955996 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:03:46.957160 (+ 1164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:46.957455 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:46.957455 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:46.957456 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:46.957503 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:46.957629 (+ 126us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:46.957630 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:46.957630 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:46.957782 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:03:46.957858 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 21:03:46.957930 (+ 72us) server_negotiation.cc:299] Negotiation successful
0812 21:03:46.958031 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
W20260812 21:03:46.958384 17858 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:39714
I20260812 21:03:46.959045 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 17974
I20260812 21:03:46.974012 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 18220
I20260812 21:03:46.989094 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 18447
I20260812 21:03:47.003752 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 17802
2026-08-12T21:03:47Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2068 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568607354921-12282-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 21:03:47 dist-test-slave-vpvm krb5kdc[18691](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 21:03:47 dist-test-slave-vpvm krb5kdc[18691](info): set up 2 sockets
Aug 12 21:03:47 dist-test-slave-vpvm krb5kdc[18691](info): commencing operation
krb5kdc: starting...
W20260812 21:03:48.187548 12282 mini_kdc.cc:121] Time spent starting KDC: real 1.154s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568607354921-12282-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568607354921-12282-0/krb5kdc/test-user.keytab.
Aug 12 21:03:48 dist-test-slave-vpvm krb5kdc[18691](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568628, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T21:03:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T21:03:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.254.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.254.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.254.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.190.keytab.
Entry for principal kudu/127.11.254.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.190.keytab.
I20260812 21:03:48.317837 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568607354921-12282-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568607354921-12282-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568607354921-12282-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.254.190:43713
--webserver_interface=127.11.254.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.190.keytab
--principal=kudu/127.11.254.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.254.148:42059
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.254.190:43713
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568607354921-12282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568607354921-12282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568607354921-12282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:03:48.401183 18714 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:03:48.401412 18714 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:03:48.401444 18714 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:03:48.402654 18714 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 21:03:48.402697 18714 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:03:48.402715 18714 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:03:48.402732 18714 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 21:03:48.402746 18714 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 21:03:48.404296 18714 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.254.148:42059
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568607354921-12282-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568607354921-12282-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.254.190:43713
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.190.keytab
--principal=kudu/127.11.254.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.254.190:43713
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.254.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18714
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568607354921-12282-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:03:48.404551 18714 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:03:48.404762 18714 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 21:03:48.409585 18714 server_base.cc:1034] running on GCE node
W20260812 21:03:48.409569 18731 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:03:48.409561 18726 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:03:48.409647 18723 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:03:48.410015 18714 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:03:48.410303 18714 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:03:48.411478 18714 hybrid_clock.cc:648] HybridClock initialized: now 1786568628411459 us; error 33 us; skew 500 ppm
Aug 12 21:03:48 dist-test-slave-vpvm krb5kdc[18691](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568628, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:03:48.414059 18714 init.cc:377] Logged in from keytab as kudu/127.11.254.190@KRBTEST.COM (short username kudu)
I20260812 21:03:48.414528 18714 webserver.cc:460] Webserver started at http://127.11.254.190:45619/ using document root <none> and password file <none>
I20260812 21:03:48.414732 18714 fs_manager.cc:359] Metadata directory not provided
I20260812 21:03:48.414772 18714 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:03:48.414868 18714 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:03:48.415951 18714 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568607354921-12282-0/minicluster-data/master-0/data/instance:
uuid: "3ce0710551f246a9ba8dd1eacccd70e8"
format_stamp: "Formatted at 2026-08-12 21:03:48 on dist-test-slave-vpvm"
server_key: "b042835c5f6b5664c4e2c40f3222ad90"
server_key_iv: "b48472a5e7f8cb630eb18c9fedbe0e25"
server_key_version: "encryptionkey@0"
I20260812 21:03:48.416273 18714 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568607354921-12282-0/minicluster-data/master-0/wal/instance:
uuid: "3ce0710551f246a9ba8dd1eacccd70e8"
format_stamp: "Formatted at 2026-08-12 21:03:48 on dist-test-slave-vpvm"
server_key: "b042835c5f6b5664c4e2c40f3222ad90"
server_key_iv: "b48472a5e7f8cb630eb18c9fedbe0e25"
server_key_version: "encryptionkey@0"
I20260812 21:03:48.417481 18714 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 21:03:48.418219 18741 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:03:48.418387 18714 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 21:03:48.418467 18714 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568607354921-12282-0/minicluster-data/master-0/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568607354921-12282-0/minicluster-data/master-0/wal
uuid: "3ce0710551f246a9ba8dd1eacccd70e8"
format_stamp: "Formatted at 2026-08-12 21:03:48 on dist-test-slave-vpvm"
server_key: "b042835c5f6b5664c4e2c40f3222ad90"
server_key_iv: "b48472a5e7f8cb630eb18c9fedbe0e25"
server_key_version: "encryptionkey@0"
I20260812 21:03:48.418579 18714 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568607354921-12282-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568607354921-12282-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568607354921-12282-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:03:48.428406 18714 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:03:48.430337 18714 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:03:48.430473 18714 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:03:48.434440 18714 rpc_server.cc:307] RPC server started. Bound to: 127.11.254.190:43713
I20260812 21:03:48.434486 18838 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.254.190:43713 every 8 connection(s)
I20260812 21:03:48.434973 18714 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
I20260812 21:03:48.435711 18842 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:03:48.438086 18842 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ce0710551f246a9ba8dd1eacccd70e8: Bootstrap starting.
I20260812 21:03:48.438661 18842 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3ce0710551f246a9ba8dd1eacccd70e8: Neither blocks nor log segments found. Creating new log.
I20260812 21:03:48.438932 18842 log.cc:824] T 00000000000000000000000000000000 P 3ce0710551f246a9ba8dd1eacccd70e8: Log is configured to *not* fsync() on all Append() calls
I20260812 21:03:48.439637 18842 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ce0710551f246a9ba8dd1eacccd70e8: No bootstrap required, opened a new log
I20260812 21:03:48.441215 18842 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3ce0710551f246a9ba8dd1eacccd70e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ce0710551f246a9ba8dd1eacccd70e8" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 43713 } }
I20260812 21:03:48.441361 18842 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3ce0710551f246a9ba8dd1eacccd70e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:03:48.441393 18842 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3ce0710551f246a9ba8dd1eacccd70e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce0710551f246a9ba8dd1eacccd70e8, State: Initialized, Role: FOLLOWER
I20260812 21:03:48.441548 18842 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3ce0710551f246a9ba8dd1eacccd70e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ce0710551f246a9ba8dd1eacccd70e8" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 43713 } }
I20260812 21:03:48.441630 18842 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3ce0710551f246a9ba8dd1eacccd70e8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 21:03:48.441671 18842 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3ce0710551f246a9ba8dd1eacccd70e8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 21:03:48.441718 18842 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3ce0710551f246a9ba8dd1eacccd70e8 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:03:48.442673 18842 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3ce0710551f246a9ba8dd1eacccd70e8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ce0710551f246a9ba8dd1eacccd70e8" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 43713 } }
I20260812 21:03:48.442777 18842 leader_election.cc:302] T 00000000000000000000000000000000 P 3ce0710551f246a9ba8dd1eacccd70e8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3ce0710551f246a9ba8dd1eacccd70e8; no voters:
I20260812 21:03:48.442935 18842 leader_election.cc:288] T 00000000000000000000000000000000 P 3ce0710551f246a9ba8dd1eacccd70e8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 21:03:48.443037 18851 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3ce0710551f246a9ba8dd1eacccd70e8 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:03:48.443209 18851 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3ce0710551f246a9ba8dd1eacccd70e8 [term 1 LEADER]: Becoming Leader. State: Replica: 3ce0710551f246a9ba8dd1eacccd70e8, State: Running, Role: LEADER
I20260812 21:03:48.443245 18842 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3ce0710551f246a9ba8dd1eacccd70e8 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 21:03:48.443300 18851 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3ce0710551f246a9ba8dd1eacccd70e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ce0710551f246a9ba8dd1eacccd70e8" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 43713 } }
I20260812 21:03:48.443632 18852 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ce0710551f246a9ba8dd1eacccd70e8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3ce0710551f246a9ba8dd1eacccd70e8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ce0710551f246a9ba8dd1eacccd70e8" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 43713 } } }
I20260812 21:03:48.443687 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 18714
I20260812 21:03:48.443709 18852 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ce0710551f246a9ba8dd1eacccd70e8 [sys.catalog]: This master's current role is: LEADER
I20260812 21:03:48.443787 12282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568607354921-12282-0/minicluster-data/master-0/wal/instance
I20260812 21:03:48.443872 18853 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ce0710551f246a9ba8dd1eacccd70e8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3ce0710551f246a9ba8dd1eacccd70e8. Latest consensus state: current_term: 1 leader_uuid: "3ce0710551f246a9ba8dd1eacccd70e8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ce0710551f246a9ba8dd1eacccd70e8" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 43713 } } }
I20260812 21:03:48.443919 18853 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ce0710551f246a9ba8dd1eacccd70e8 [sys.catalog]: This master's current role is: LEADER
I20260812 21:03:48.443982 12282 external_mini_cluster.cc:1428] Setting key 9a68a97675417c4eeec8ee25180887ba
I20260812 21:03:48.444555 18859 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 21:03:48.445317 18859 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 21:03:48.446805 18859 catalog_manager.cc:1347] Generated new cluster ID: 6933de869dd946708cac26f0704c2978
I20260812 21:03:48.446856 18859 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 21:03:48 dist-test-slave-vpvm krb5kdc[18691](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568628, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.254.190@KRBTEST.COM
I20260812 21:03:48.458916 18884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:48.444675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60212 (local address 127.11.254.190:43713)
0812 21:03:48.444833 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0812 21:03:48.444843 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 21:03:48.444874 (+ 31us) server_negotiation.cc:407] Connection header received
0812 21:03:48.444995 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:03:48.445003 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:03:48.445062 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:03:48.445214 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:03:48.446017 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:48.450583 (+ 4566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:48.451686 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:48.452345 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:48.452441 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:48.455296 (+ 2855us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:03:48.455300 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:03:48.455301 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:03:48.455316 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:03:48.457243 (+ 1927us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:48.457619 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:48.457622 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:48.457623 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:48.457737 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:48.457908 (+ 171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:48.457908 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:48.457909 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:48.458268 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:03:48.458425 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0812 21:03:48.458513 (+ 88us) server_negotiation.cc:299] Negotiation successful
0812 21:03:48.458660 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":65,"threads_started":1}
I20260812 21:03:48.477460 18859 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 21:03:48.478129 18859 catalog_manager.cc:1504] Loading token signing keys...
I20260812 21:03:48.481830 18859 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3ce0710551f246a9ba8dd1eacccd70e8: Generated new TSK 0
I20260812 21:03:48.482015 18859 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.254.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.254.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.254.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.129.keytab.
Entry for principal kudu/127.11.254.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.129.keytab.
I20260812 21:03:48.513850 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568607354921-12282-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568607354921-12282-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568607354921-12282-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568607354921-12282-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.254.129:0
--local_ip_for_outbound_sockets=127.11.254.129
--webserver_interface=127.11.254.129
--webserver_port=0
--tserver_master_addrs=127.11.254.190:43713
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.129.keytab
--principal=kudu/127.11.254.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.254.148:42059
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568607354921-12282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568607354921-12282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568607354921-12282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:03:48.596717 18890 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:03:48.596920 18890 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:03:48.596942 18890 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:03:48.598095 18890 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:03:48.598127 18890 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:03:48.598151 18890 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.254.129
I20260812 21:03:48.599669 18890 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.254.148:42059
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568607354921-12282-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568607354921-12282-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.129.keytab
--principal=kudu/127.11.254.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.254.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568607354921-12282-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.254.129
--webserver_port=0
--tserver_master_addrs=127.11.254.190:43713
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18890
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.254.129
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568607354921-12282-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:03:48.599870 18890 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:03:48.600055 18890 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:03:48.605084 18899 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:03:48.605084 18900 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:03:48.605088 18903 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:03:48.605356 18890 server_base.cc:1034] running on GCE node
I20260812 21:03:48.605593 18890 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:03:48.605798 18890 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:03:48.606956 18890 hybrid_clock.cc:648] HybridClock initialized: now 1786568628606938 us; error 30 us; skew 500 ppm
Aug 12 21:03:48 dist-test-slave-vpvm krb5kdc[18691](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568628, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:03:48.609545 18890 init.cc:377] Logged in from keytab as kudu/127.11.254.129@KRBTEST.COM (short username kudu)
I20260812 21:03:48.610024 18890 webserver.cc:460] Webserver started at http://127.11.254.129:35801/ using document root <none> and password file <none>
I20260812 21:03:48.610262 18890 fs_manager.cc:359] Metadata directory not provided
I20260812 21:03:48.610313 18890 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:03:48.610442 18890 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:03:48.611526 18890 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568607354921-12282-0/minicluster-data/ts-0/data/instance:
uuid: "0f63ea05211b4f31a1f07cb841707f6c"
format_stamp: "Formatted at 2026-08-12 21:03:48 on dist-test-slave-vpvm"
server_key: "515c593580543756deabecd272a8c526"
server_key_iv: "2450ce161be16cbb9921ff1acb8d075f"
server_key_version: "encryptionkey@0"
I20260812 21:03:48.611912 18890 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568607354921-12282-0/minicluster-data/ts-0/wal/instance:
uuid: "0f63ea05211b4f31a1f07cb841707f6c"
format_stamp: "Formatted at 2026-08-12 21:03:48 on dist-test-slave-vpvm"
server_key: "515c593580543756deabecd272a8c526"
server_key_iv: "2450ce161be16cbb9921ff1acb8d075f"
server_key_version: "encryptionkey@0"
I20260812 21:03:48.613245 18890 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 21:03:48.614017 18911 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:03:48.614176 18890 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 21:03:48.614264 18890 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568607354921-12282-0/minicluster-data/ts-0/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568607354921-12282-0/minicluster-data/ts-0/wal
uuid: "0f63ea05211b4f31a1f07cb841707f6c"
format_stamp: "Formatted at 2026-08-12 21:03:48 on dist-test-slave-vpvm"
server_key: "515c593580543756deabecd272a8c526"
server_key_iv: "2450ce161be16cbb9921ff1acb8d075f"
server_key_version: "encryptionkey@0"
I20260812 21:03:48.614336 18890 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568607354921-12282-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568607354921-12282-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568607354921-12282-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:03:48.629664 18890 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:03:48.631942 18890 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:03:48.632094 18890 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:03:48.632321 18890 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:03:48.632647 18890 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:03:48.632682 18890 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:03:48.632714 18890 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:03:48.632738 18890 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:03:48.638386 18890 rpc_server.cc:307] RPC server started. Bound to: 127.11.254.129:35993
I20260812 21:03:48.638435 19144 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.254.129:35993 every 8 connection(s)
I20260812 21:03:48.638785 18890 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568607354921-12282-0/minicluster-data/ts-0/data/info.pb
I20260812 21:03:48.639493 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 18890
I20260812 21:03:48.639608 12282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568607354921-12282-0/minicluster-data/ts-0/wal/instance
I20260812 21:03:48.639837 12282 external_mini_cluster.cc:1428] Setting key 7b76731faa7e1d7cf481c6f85882ef0c
Aug 12 21:03:48 dist-test-slave-vpvm krb5kdc[18691](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568628, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.129@KRBTEST.COM for kudu/127.11.254.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.11.254.130@KRBTEST.COM; defaulting to no policy
I20260812 21:03:48.649901 18884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:48.639615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.129:41917 (local address 127.11.254.190:43713)
0812 21:03:48.639887 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0812 21:03:48.639889 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:03:48.639904 (+ 15us) server_negotiation.cc:407] Connection header received
0812 21:03:48.641178 (+ 1274us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:03:48.641179 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:03:48.641196 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:03:48.641267 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:03:48.641892 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:48.642562 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:48.643982 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:48.644290 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:48.644361 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:48.646701 (+ 2340us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:03:48.646703 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:03:48.646703 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:03:48.646712 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:03:48.648409 (+ 1697us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:48.648920 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:48.648921 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:48.648921 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:48.648985 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:48.649248 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:48.649248 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:48.649248 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:48.649367 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:03:48.649429 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0812 21:03:48.649709 (+ 280us) server_negotiation.cc:299] Negotiation successful
0812 21:03:48.649806 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":237}
I20260812 21:03:48.650111 19151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:48.639613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:43713 (local address 127.11.254.129:41917)
0812 21:03:48.639780 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0812 21:03:48.639821 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 21:03:48.641050 (+ 1229us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:03:48.641328 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:03:48.641332 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:03:48.641601 (+ 269us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:03:48.641802 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:48.641806 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:48.642679 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:48.642681 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:48.643906 (+ 1225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:48.643909 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:48.644368 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:48.644369 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:48.644488 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:48.645037 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:03:48.645045 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:03:48.646610 (+ 1565us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:03:48.648509 (+ 1899us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:03:48.648518 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:03:48.648520 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:03:48.648821 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:03:48.649052 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:03:48.649053 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:03:48.649053 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:03:48.649188 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:03:48.649423 (+ 235us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:03:48.649431 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:03:48.649583 (+ 152us) client_negotiation.cc:769] Sending connection context
0812 21:03:48.649675 (+ 92us) client_negotiation.cc:240] Negotiation successful
0812 21:03:48.649842 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":93,"threads_started":1}
I20260812 21:03:48.650616 19149 heartbeater.cc:347] Connected to a master server at 127.11.254.190:43713
I20260812 21:03:48.650707 19149 heartbeater.cc:464] Registering TS with master...
I20260812 21:03:48.650861 19149 heartbeater.cc:511] Master 127.11.254.190:43713 requested a full tablet report, sending...
I20260812 21:03:48.651371 18769 ts_manager.cc:194] Registered new tserver with Master: 0f63ea05211b4f31a1f07cb841707f6c (127.11.254.129:35993)
I20260812 21:03:48.652017 18769 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.254.129@KRBTEST.COM'} at 127.11.254.129:41917
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.254.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.254.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.130.keytab.
Entry for principal kudu/127.11.254.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.130.keytab.
I20260812 21:03:48.669235 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568607354921-12282-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568607354921-12282-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568607354921-12282-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568607354921-12282-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.254.130:0
--local_ip_for_outbound_sockets=127.11.254.130
--webserver_interface=127.11.254.130
--webserver_port=0
--tserver_master_addrs=127.11.254.190:43713
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.130.keytab
--principal=kudu/127.11.254.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.254.148:42059
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568607354921-12282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568607354921-12282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568607354921-12282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:03:48.752125 19163 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:03:48.752352 19163 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:03:48.752375 19163 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:03:48.753506 19163 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:03:48.753543 19163 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:03:48.753568 19163 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.254.130
I20260812 21:03:48.755157 19163 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.254.148:42059
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568607354921-12282-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568607354921-12282-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.130.keytab
--principal=kudu/127.11.254.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.254.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568607354921-12282-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.254.130
--webserver_port=0
--tserver_master_addrs=127.11.254.190:43713
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19163
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.254.130
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568607354921-12282-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:03:48.755371 19163 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:03:48.755578 19163 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:03:48.760771 19174 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:03:48.760774 19179 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:03:48.761080 19163 server_base.cc:1034] running on GCE node
W20260812 21:03:48.761186 19175 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:03:48.761405 19163 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:03:48.761656 19163 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:03:48.762811 19163 hybrid_clock.cc:648] HybridClock initialized: now 1786568628762795 us; error 30 us; skew 500 ppm
Aug 12 21:03:48 dist-test-slave-vpvm krb5kdc[18691](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568628, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:03:48.765393 19163 init.cc:377] Logged in from keytab as kudu/127.11.254.130@KRBTEST.COM (short username kudu)
I20260812 21:03:48.765838 19163 webserver.cc:460] Webserver started at http://127.11.254.130:43663/ using document root <none> and password file <none>
I20260812 21:03:48.766079 19163 fs_manager.cc:359] Metadata directory not provided
I20260812 21:03:48.766132 19163 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:03:48.766266 19163 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:03:48.767331 19163 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568607354921-12282-0/minicluster-data/ts-1/data/instance:
uuid: "eb8b9633fede44ccad9e4ba1f9aa3ba5"
format_stamp: "Formatted at 2026-08-12 21:03:48 on dist-test-slave-vpvm"
server_key: "0c6e6093824c696def0fd612438e0a97"
server_key_iv: "f18c4aaabfeaef9da103f091586bcfdd"
server_key_version: "encryptionkey@0"
I20260812 21:03:48.767685 19163 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568607354921-12282-0/minicluster-data/ts-1/wal/instance:
uuid: "eb8b9633fede44ccad9e4ba1f9aa3ba5"
format_stamp: "Formatted at 2026-08-12 21:03:48 on dist-test-slave-vpvm"
server_key: "0c6e6093824c696def0fd612438e0a97"
server_key_iv: "f18c4aaabfeaef9da103f091586bcfdd"
server_key_version: "encryptionkey@0"
I20260812 21:03:48.768998 19163 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 21:03:48.769634 19198 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:03:48.769806 19163 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 21:03:48.769891 19163 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568607354921-12282-0/minicluster-data/ts-1/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568607354921-12282-0/minicluster-data/ts-1/wal
uuid: "eb8b9633fede44ccad9e4ba1f9aa3ba5"
format_stamp: "Formatted at 2026-08-12 21:03:48 on dist-test-slave-vpvm"
server_key: "0c6e6093824c696def0fd612438e0a97"
server_key_iv: "f18c4aaabfeaef9da103f091586bcfdd"
server_key_version: "encryptionkey@0"
I20260812 21:03:48.769985 19163 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568607354921-12282-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568607354921-12282-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568607354921-12282-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:03:48.780845 19163 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:03:48.782702 19163 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:03:48.782840 19163 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:03:48.783048 19163 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:03:48.783360 19163 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:03:48.783393 19163 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:03:48.783422 19163 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:03:48.783445 19163 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:03:48.789009 19163 rpc_server.cc:307] RPC server started. Bound to: 127.11.254.130:38139
I20260812 21:03:48.789057 19399 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.254.130:38139 every 8 connection(s)
I20260812 21:03:48.789364 19163 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568607354921-12282-0/minicluster-data/ts-1/data/info.pb
Aug 12 21:03:48 dist-test-slave-vpvm krb5kdc[18691](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568628, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.130@KRBTEST.COM for kudu/127.11.254.190@KRBTEST.COM
I20260812 21:03:48.795375 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 19163
I20260812 21:03:48.795483 12282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568607354921-12282-0/minicluster-data/ts-1/wal/instance
I20260812 21:03:48.795696 12282 external_mini_cluster.cc:1428] Setting key 26444ab9a8664347c525fc3869a420bd
I20260812 21:03:48.798844 18884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:48.790127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.130:40753 (local address 127.11.254.190:43713)
0812 21:03:48.790199 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0812 21:03:48.790201 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:03:48.790365 (+ 164us) server_negotiation.cc:407] Connection header received
0812 21:03:48.791254 (+ 889us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:03:48.791255 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:03:48.791265 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:03:48.791314 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:03:48.791928 (+ 614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:48.792429 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:48.793371 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:48.793560 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:48.793606 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:48.795913 (+ 2307us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:03:48.795914 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:03:48.795914 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:03:48.795924 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:03:48.797329 (+ 1405us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:48.797813 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:48.797814 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:48.797815 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:48.797877 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:48.798171 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:48.798172 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:48.798173 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:48.798326 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:03:48.798396 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 21:03:48.798651 (+ 255us) server_negotiation.cc:299] Negotiation successful
0812 21:03:48.798753 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260812 21:03:48.798925 19405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:48.790089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:43713 (local address 127.11.254.130:40753)
0812 21:03:48.790247 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0812 21:03:48.790283 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0812 21:03:48.791171 (+ 888us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:03:48.791370 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:03:48.791373 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:03:48.791661 (+ 288us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:03:48.791859 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:48.791863 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:48.792492 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:48.792493 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:48.793308 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:48.793310 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:48.793618 (+ 308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:48.793619 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:48.793722 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:48.794249 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:03:48.794258 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:03:48.795794 (+ 1536us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:03:48.797405 (+ 1611us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:03:48.797414 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:03:48.797416 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:03:48.797718 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:03:48.797938 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:03:48.797939 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:03:48.797939 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:03:48.798102 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:03:48.798390 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:03:48.798394 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:03:48.798529 (+ 135us) client_negotiation.cc:769] Sending connection context
0812 21:03:48.798615 (+ 86us) client_negotiation.cc:240] Negotiation successful
0812 21:03:48.798746 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":83,"threads_started":1}
I20260812 21:03:48.799381 19401 heartbeater.cc:347] Connected to a master server at 127.11.254.190:43713
I20260812 21:03:48.799484 19401 heartbeater.cc:464] Registering TS with master...
I20260812 21:03:48.799659 19401 heartbeater.cc:511] Master 127.11.254.190:43713 requested a full tablet report, sending...
I20260812 21:03:48.800033 18769 ts_manager.cc:194] Registered new tserver with Master: eb8b9633fede44ccad9e4ba1f9aa3ba5 (127.11.254.130:38139)
I20260812 21:03:48.800488 18769 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.254.130@KRBTEST.COM'} at 127.11.254.130:40753
WARNING: no policy specified for kudu/127.11.254.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.254.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.254.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.131.keytab.
Entry for principal kudu/127.11.254.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.131.keytab.
I20260812 21:03:48.824982 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568607354921-12282-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568607354921-12282-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568607354921-12282-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568607354921-12282-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.254.131:0
--local_ip_for_outbound_sockets=127.11.254.131
--webserver_interface=127.11.254.131
--webserver_port=0
--tserver_master_addrs=127.11.254.190:43713
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.131.keytab
--principal=kudu/127.11.254.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.254.148:42059
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568607354921-12282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568607354921-12282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568607354921-12282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:03:48.906914 19409 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:03:48.907132 19409 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:03:48.907155 19409 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:03:48.908250 19409 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:03:48.908286 19409 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:03:48.908311 19409 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.254.131
I20260812 21:03:48.909770 19409 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.254.148:42059
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568607354921-12282-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568607354921-12282-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.131.keytab
--principal=kudu/127.11.254.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.254.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568607354921-12282-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.254.131
--webserver_port=0
--tserver_master_addrs=127.11.254.190:43713
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19409
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.254.131
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568607354921-12282-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:03:48.910000 19409 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:03:48.910207 19409 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:03:48.915295 19416 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:03:48.915297 19421 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:03:48.915297 19417 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:03:48.915385 19409 server_base.cc:1034] running on GCE node
I20260812 21:03:48.915776 19409 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:03:48.916029 19409 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:03:48.917192 19409 hybrid_clock.cc:648] HybridClock initialized: now 1786568628917172 us; error 35 us; skew 500 ppm
Aug 12 21:03:48 dist-test-slave-vpvm krb5kdc[18691](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568628, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:03:48.919862 19409 init.cc:377] Logged in from keytab as kudu/127.11.254.131@KRBTEST.COM (short username kudu)
I20260812 21:03:48.920316 19409 webserver.cc:460] Webserver started at http://127.11.254.131:32815/ using document root <none> and password file <none>
I20260812 21:03:48.920570 19409 fs_manager.cc:359] Metadata directory not provided
I20260812 21:03:48.920622 19409 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:03:48.920743 19409 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:03:48.921850 19409 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568607354921-12282-0/minicluster-data/ts-2/data/instance:
uuid: "9207ce4dd69649dab2b5a9e9f1192fbc"
format_stamp: "Formatted at 2026-08-12 21:03:48 on dist-test-slave-vpvm"
server_key: "2195a614c4b1023c6651400147865f6b"
server_key_iv: "a3b34c33324b52468224240e074affff"
server_key_version: "encryptionkey@0"
I20260812 21:03:48.922242 19409 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568607354921-12282-0/minicluster-data/ts-2/wal/instance:
uuid: "9207ce4dd69649dab2b5a9e9f1192fbc"
format_stamp: "Formatted at 2026-08-12 21:03:48 on dist-test-slave-vpvm"
server_key: "2195a614c4b1023c6651400147865f6b"
server_key_iv: "a3b34c33324b52468224240e074affff"
server_key_version: "encryptionkey@0"
I20260812 21:03:48.923610 19409 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 21:03:48.924376 19431 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:03:48.924576 19409 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 21:03:48.924667 19409 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568607354921-12282-0/minicluster-data/ts-2/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568607354921-12282-0/minicluster-data/ts-2/wal
uuid: "9207ce4dd69649dab2b5a9e9f1192fbc"
format_stamp: "Formatted at 2026-08-12 21:03:48 on dist-test-slave-vpvm"
server_key: "2195a614c4b1023c6651400147865f6b"
server_key_iv: "a3b34c33324b52468224240e074affff"
server_key_version: "encryptionkey@0"
I20260812 21:03:48.924729 19409 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568607354921-12282-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568607354921-12282-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568607354921-12282-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:03:48.937860 19409 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:03:48.940201 19409 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:03:48.940348 19409 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:03:48.940590 19409 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:03:48.940907 19409 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:03:48.940943 19409 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:03:48.940972 19409 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:03:48.940991 19409 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:03:48.946898 19409 rpc_server.cc:307] RPC server started. Bound to: 127.11.254.131:35579
I20260812 21:03:48.946950 19632 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.254.131:35579 every 8 connection(s)
I20260812 21:03:48.947257 19409 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568607354921-12282-0/minicluster-data/ts-2/data/info.pb
I20260812 21:03:48.950549 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 19409
I20260812 21:03:48.950680 12282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786568607354921-12282-0/minicluster-data/ts-2/wal/instance
I20260812 21:03:48.950917 12282 external_mini_cluster.cc:1428] Setting key 0bbf8c3eee9b28164c7b6a2b6dac7541
Aug 12 21:03:48 dist-test-slave-vpvm krb5kdc[18691](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568628, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.131@KRBTEST.COM for kudu/127.11.254.190@KRBTEST.COM
I20260812 21:03:48.956835 18884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:48.947933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.131:51577 (local address 127.11.254.190:43713)
0812 21:03:48.948005 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0812 21:03:48.948008 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 21:03:48.948200 (+ 192us) server_negotiation.cc:407] Connection header received
0812 21:03:48.949163 (+ 963us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:03:48.949165 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:03:48.949177 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:03:48.949243 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:03:48.949884 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:48.950509 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:48.951827 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:48.952116 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:48.952175 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:48.954299 (+ 2124us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:03:48.954300 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:03:48.954301 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:03:48.954308 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:03:48.955596 (+ 1288us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:48.955996 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:48.955996 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:48.955997 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:48.956042 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:48.956223 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:48.956224 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:48.956224 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:48.956325 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:03:48.956371 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0812 21:03:48.956698 (+ 327us) server_negotiation.cc:299] Negotiation successful
0812 21:03:48.956764 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260812 21:03:48.957072 19636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:48.947918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:43713 (local address 127.11.254.131:51577)
0812 21:03:48.948076 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0812 21:03:48.948118 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 21:03:48.949058 (+ 940us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:03:48.949303 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:03:48.949306 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:03:48.949573 (+ 267us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:03:48.949780 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:48.949785 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:48.950641 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:48.950643 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:48.951748 (+ 1105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:48.951751 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:48.952207 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:48.952208 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:48.952333 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:48.952800 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:03:48.952809 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:03:48.954212 (+ 1403us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:03:48.955681 (+ 1469us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:03:48.955688 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:03:48.955690 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:03:48.955936 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:03:48.956086 (+ 150us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:03:48.956087 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:03:48.956087 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:03:48.956182 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:03:48.956416 (+ 234us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:03:48.956423 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:03:48.956568 (+ 145us) client_negotiation.cc:769] Sending connection context
0812 21:03:48.956670 (+ 102us) client_negotiation.cc:240] Negotiation successful
0812 21:03:48.956813 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":78,"threads_started":1}
I20260812 21:03:48.957643 19634 heartbeater.cc:347] Connected to a master server at 127.11.254.190:43713
I20260812 21:03:48.957744 19634 heartbeater.cc:464] Registering TS with master...
I20260812 21:03:48.957984 19634 heartbeater.cc:511] Master 127.11.254.190:43713 requested a full tablet report, sending...
I20260812 21:03:48.958391 18769 ts_manager.cc:194] Registered new tserver with Master: 9207ce4dd69649dab2b5a9e9f1192fbc (127.11.254.131:35579)
I20260812 21:03:48.958792 18769 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.254.131@KRBTEST.COM'} at 127.11.254.131:51577
I20260812 21:03:48.963018 12282 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 12 21:03:48 dist-test-slave-vpvm krb5kdc[18691](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568628, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 21:03:48 dist-test-slave-vpvm krb5kdc[18691](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568628, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.254.129@KRBTEST.COM
I20260812 21:03:48.987238 19643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:48.979411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58100 (local address 127.11.254.129:35993)
0812 21:03:48.979637 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0812 21:03:48.979643 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 21:03:48.979654 (+ 11us) server_negotiation.cc:407] Connection header received
0812 21:03:48.979683 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:03:48.979685 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:03:48.979700 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:03:48.979777 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:03:48.980344 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:48.981113 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:48.981941 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:48.982322 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:48.982386 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:48.984578 (+ 2192us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:03:48.984588 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:03:48.984588 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:03:48.984595 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:03:48.986081 (+ 1486us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:48.986430 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:48.986434 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:48.986434 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:48.986508 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:48.986699 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:48.986700 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:48.986701 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:48.986865 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:03:48.987018 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0812 21:03:48.987055 (+ 37us) server_negotiation.cc:299] Negotiation successful
0812 21:03:48.987127 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":191,"thread_start_us":108,"threads_started":1}
Aug 12 21:03:48 dist-test-slave-vpvm krb5kdc[18691](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568628, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.254.190@KRBTEST.COM
I20260812 21:03:48.995332 18884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:48.988612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52898 (local address 127.11.254.190:43713)
0812 21:03:48.988670 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0812 21:03:48.988672 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:03:48.988680 (+ 8us) server_negotiation.cc:407] Connection header received
0812 21:03:48.988702 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:03:48.988702 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:03:48.988712 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:03:48.988755 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:03:48.989190 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:48.989699 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:48.990380 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:48.990590 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:48.990647 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:48.992787 (+ 2140us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:03:48.992788 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:03:48.992789 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:03:48.992796 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:03:48.994187 (+ 1391us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:48.994551 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:48.994552 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:48.994553 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:48.994611 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:48.994846 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:48.994847 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:48.994847 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:48.995025 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:03:48.995106 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0812 21:03:48.995183 (+ 77us) server_negotiation.cc:299] Negotiation successful
0812 21:03:48.995256 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
W20260812 21:03:48.995625 18769 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52898
I20260812 21:03:48.995991 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 18890
I20260812 21:03:49.017508 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 19163
I20260812 21:03:49.027199 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 19409
I20260812 21:03:49.040478 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 18714
2026-08-12T21:03:49Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2032 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568607354921-12282-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 21:03:49 dist-test-slave-vpvm krb5kdc[19654](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 21:03:49 dist-test-slave-vpvm krb5kdc[19654](info): set up 2 sockets
Aug 12 21:03:49 dist-test-slave-vpvm krb5kdc[19654](info): commencing operation
krb5kdc: starting...
W20260812 21:03:50.218906 12282 mini_kdc.cc:121] Time spent starting KDC: real 1.153s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568607354921-12282-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568607354921-12282-0/krb5kdc/test-user.keytab.
Aug 12 21:03:50 dist-test-slave-vpvm krb5kdc[19654](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568630, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T21:03:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T21:03:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.254.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.254.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.254.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.190.keytab.
Entry for principal kudu/127.11.254.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.190.keytab.
I20260812 21:03:50.351011 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568607354921-12282-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568607354921-12282-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568607354921-12282-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.254.190:45239
--webserver_interface=127.11.254.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.190.keytab
--principal=kudu/127.11.254.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.254.148:38427
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.254.190:45239
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568607354921-12282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568607354921-12282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568607354921-12282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:03:50.433701 19679 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:03:50.433930 19679 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:03:50.433985 19679 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:03:50.435196 19679 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 21:03:50.435245 19679 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:03:50.435257 19679 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:03:50.435269 19679 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 21:03:50.435281 19679 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 21:03:50.436781 19679 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.254.148:38427
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568607354921-12282-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568607354921-12282-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.254.190:45239
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.190.keytab
--principal=kudu/127.11.254.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.254.190:45239
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.254.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19679
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568607354921-12282-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:03:50.436983 19679 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:03:50.437156 19679 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:03:50.441984 19693 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:03:50.442000 19699 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:03:50.442117 19679 server_base.cc:1034] running on GCE node
W20260812 21:03:50.442198 19691 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:03:50.442461 19679 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:03:50.442756 19679 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:03:50.443957 19679 hybrid_clock.cc:648] HybridClock initialized: now 1786568630443944 us; error 39 us; skew 500 ppm
Aug 12 21:03:50 dist-test-slave-vpvm krb5kdc[19654](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568630, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:03:50.450814 19679 init.cc:377] Logged in from keytab as kudu/127.11.254.190@KRBTEST.COM (short username kudu)
I20260812 21:03:50.451182 19679 fs_manager.cc:359] Metadata directory not provided
I20260812 21:03:50.451238 19679 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:03:50.451367 19679 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:03:50.452500 19679 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568607354921-12282-0/minicluster-data/master-0/data/instance:
uuid: "2c39843ab1224081ab700e6a3b53cb5a"
format_stamp: "Formatted at 2026-08-12 21:03:50 on dist-test-slave-vpvm"
server_key: "669153ce0aa35cb72094b4a576fb5bb3"
server_key_iv: "8d14f7f2a349dfbfefad3cb11eac06e1"
server_key_version: "encryptionkey@0"
I20260812 21:03:50.452859 19679 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568607354921-12282-0/minicluster-data/master-0/wal/instance:
uuid: "2c39843ab1224081ab700e6a3b53cb5a"
format_stamp: "Formatted at 2026-08-12 21:03:50 on dist-test-slave-vpvm"
server_key: "669153ce0aa35cb72094b4a576fb5bb3"
server_key_iv: "8d14f7f2a349dfbfefad3cb11eac06e1"
server_key_version: "encryptionkey@0"
I20260812 21:03:50.454290 19679 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 21:03:50.455144 19707 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:03:50.455304 19679 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 21:03:50.455395 19679 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568607354921-12282-0/minicluster-data/master-0/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568607354921-12282-0/minicluster-data/master-0/wal
uuid: "2c39843ab1224081ab700e6a3b53cb5a"
format_stamp: "Formatted at 2026-08-12 21:03:50 on dist-test-slave-vpvm"
server_key: "669153ce0aa35cb72094b4a576fb5bb3"
server_key_iv: "8d14f7f2a349dfbfefad3cb11eac06e1"
server_key_version: "encryptionkey@0"
I20260812 21:03:50.455531 19679 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568607354921-12282-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568607354921-12282-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568607354921-12282-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:03:50.467092 19679 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:03:50.469296 19679 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:03:50.469457 19679 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:03:50.473680 19679 rpc_server.cc:307] RPC server started. Bound to: 127.11.254.190:45239
I20260812 21:03:50.473706 19805 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.254.190:45239 every 8 connection(s)
I20260812 21:03:50.474031 19679 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
I20260812 21:03:50.475153 19806 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:03:50.476649 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 19679
I20260812 21:03:50.476763 12282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568607354921-12282-0/minicluster-data/master-0/wal/instance
I20260812 21:03:50.476946 12282 external_mini_cluster.cc:1428] Setting key 4cbb79e42089769d0abe9e8f5cd17199
I20260812 21:03:50.477855 19806 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2c39843ab1224081ab700e6a3b53cb5a: Bootstrap starting.
I20260812 21:03:50.478487 19806 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2c39843ab1224081ab700e6a3b53cb5a: Neither blocks nor log segments found. Creating new log.
I20260812 21:03:50.478813 19806 log.cc:824] T 00000000000000000000000000000000 P 2c39843ab1224081ab700e6a3b53cb5a: Log is configured to *not* fsync() on all Append() calls
I20260812 21:03:50.479548 19806 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2c39843ab1224081ab700e6a3b53cb5a: No bootstrap required, opened a new log
I20260812 21:03:50.481606 19806 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2c39843ab1224081ab700e6a3b53cb5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c39843ab1224081ab700e6a3b53cb5a" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 45239 } }
I20260812 21:03:50.481783 19806 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2c39843ab1224081ab700e6a3b53cb5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:03:50.481820 19806 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2c39843ab1224081ab700e6a3b53cb5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c39843ab1224081ab700e6a3b53cb5a, State: Initialized, Role: FOLLOWER
I20260812 21:03:50.481973 19806 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2c39843ab1224081ab700e6a3b53cb5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c39843ab1224081ab700e6a3b53cb5a" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 45239 } }
I20260812 21:03:50.482064 19806 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2c39843ab1224081ab700e6a3b53cb5a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 21:03:50.482096 19806 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2c39843ab1224081ab700e6a3b53cb5a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 21:03:50.482169 19806 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2c39843ab1224081ab700e6a3b53cb5a [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:03:50.482726 19806 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2c39843ab1224081ab700e6a3b53cb5a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c39843ab1224081ab700e6a3b53cb5a" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 45239 } }
I20260812 21:03:50.482826 19806 leader_election.cc:302] T 00000000000000000000000000000000 P 2c39843ab1224081ab700e6a3b53cb5a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2c39843ab1224081ab700e6a3b53cb5a; no voters:
I20260812 21:03:50.482991 19806 leader_election.cc:288] T 00000000000000000000000000000000 P 2c39843ab1224081ab700e6a3b53cb5a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 21:03:50.483070 19815 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2c39843ab1224081ab700e6a3b53cb5a [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:03:50.483261 19806 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2c39843ab1224081ab700e6a3b53cb5a [sys.catalog]: configured and running, proceeding with master startup.
I20260812 21:03:50.483309 19815 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2c39843ab1224081ab700e6a3b53cb5a [term 1 LEADER]: Becoming Leader. State: Replica: 2c39843ab1224081ab700e6a3b53cb5a, State: Running, Role: LEADER
I20260812 21:03:50.483407 19815 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2c39843ab1224081ab700e6a3b53cb5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c39843ab1224081ab700e6a3b53cb5a" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 45239 } }
Aug 12 21:03:50 dist-test-slave-vpvm krb5kdc[19654](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568630, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.254.190@KRBTEST.COM
I20260812 21:03:50.483767 19815 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2c39843ab1224081ab700e6a3b53cb5a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2c39843ab1224081ab700e6a3b53cb5a. Latest consensus state: current_term: 1 leader_uuid: "2c39843ab1224081ab700e6a3b53cb5a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c39843ab1224081ab700e6a3b53cb5a" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 45239 } } }
I20260812 21:03:50.483829 19815 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2c39843ab1224081ab700e6a3b53cb5a [sys.catalog]: This master's current role is: LEADER
I20260812 21:03:50.483994 19816 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2c39843ab1224081ab700e6a3b53cb5a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2c39843ab1224081ab700e6a3b53cb5a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c39843ab1224081ab700e6a3b53cb5a" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 45239 } } }
I20260812 21:03:50.484086 19816 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2c39843ab1224081ab700e6a3b53cb5a [sys.catalog]: This master's current role is: LEADER
I20260812 21:03:50.484047 19831 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 21:03:50.484681 19831 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 21:03:50.486545 19831 catalog_manager.cc:1347] Generated new cluster ID: 121395b4c8fb493485854a67d9a10da9
I20260812 21:03:50.486590 19831 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 21:03:50.488341 19813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:50.477492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33370 (local address 127.11.254.190:45239)
0812 21:03:50.477689 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0812 21:03:50.477698 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 21:03:50.477723 (+ 25us) server_negotiation.cc:407] Connection header received
0812 21:03:50.478203 (+ 480us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:03:50.478211 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:03:50.478261 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:03:50.478376 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:03:50.479146 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:50.480172 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:50.481199 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:50.481598 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:50.481672 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:50.484374 (+ 2702us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:03:50.484378 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:03:50.484379 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:03:50.484392 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:03:50.486345 (+ 1953us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:50.486827 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:50.486830 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:50.486830 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:50.486930 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:50.487101 (+ 171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:50.487102 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:50.487102 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:50.487378 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:03:50.487515 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0812 21:03:50.487999 (+ 484us) server_negotiation.cc:299] Negotiation successful
0812 21:03:50.488133 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":103,"threads_started":1}
I20260812 21:03:50.496019 19831 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 21:03:50.496548 19831 catalog_manager.cc:1504] Loading token signing keys...
I20260812 21:03:50.502132 19831 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2c39843ab1224081ab700e6a3b53cb5a: Generated new TSK 0
I20260812 21:03:50.502351 19831 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.254.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.254.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.254.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.129.keytab.
Entry for principal kudu/127.11.254.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.129.keytab.
W20260812 21:03:51.154894 12282 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.11.254.129: real 0.649s user 0.000s sys 0.004s
I20260812 21:03:51.155296 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568607354921-12282-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568607354921-12282-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568607354921-12282-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568607354921-12282-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.254.129:0
--local_ip_for_outbound_sockets=127.11.254.129
--webserver_interface=127.11.254.129
--webserver_port=0
--tserver_master_addrs=127.11.254.190:45239
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.129.keytab
--principal=kudu/127.11.254.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.254.148:38427
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568607354921-12282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568607354921-12282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568607354921-12282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:03:51.239619 19851 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:03:51.239843 19851 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:03:51.239869 19851 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:03:51.241155 19851 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:03:51.241215 19851 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:03:51.241245 19851 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.254.129
I20260812 21:03:51.243023 19851 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.254.148:38427
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568607354921-12282-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568607354921-12282-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.129.keytab
--principal=kudu/127.11.254.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.254.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568607354921-12282-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.254.129
--webserver_port=0
--tserver_master_addrs=127.11.254.190:45239
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19851
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.254.129
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568607354921-12282-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:03:51.243320 19851 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:03:51.243578 19851 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:03:51.248864 19866 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:03:51.248919 19863 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:03:51.248955 19862 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:03:51.249156 19851 server_base.cc:1034] running on GCE node
I20260812 21:03:51.249357 19851 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:03:51.249588 19851 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:03:51.250746 19851 hybrid_clock.cc:648] HybridClock initialized: now 1786568631250731 us; error 33 us; skew 500 ppm
Aug 12 21:03:51 dist-test-slave-vpvm krb5kdc[19654](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568631, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:03:51.253436 19851 init.cc:377] Logged in from keytab as kudu/127.11.254.129@KRBTEST.COM (short username kudu)
I20260812 21:03:51.253791 19851 fs_manager.cc:359] Metadata directory not provided
I20260812 21:03:51.253842 19851 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:03:51.253939 19851 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:03:51.255051 19851 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568607354921-12282-0/minicluster-data/ts-0/data/instance:
uuid: "94daf355993f4a3dbb64c14e00465548"
format_stamp: "Formatted at 2026-08-12 21:03:51 on dist-test-slave-vpvm"
server_key: "7d0b6d8f1570dad53003bb939916b159"
server_key_iv: "99594ffafc02b2bdbca85ad741e4239f"
server_key_version: "encryptionkey@0"
I20260812 21:03:51.255391 19851 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568607354921-12282-0/minicluster-data/ts-0/wal/instance:
uuid: "94daf355993f4a3dbb64c14e00465548"
format_stamp: "Formatted at 2026-08-12 21:03:51 on dist-test-slave-vpvm"
server_key: "7d0b6d8f1570dad53003bb939916b159"
server_key_iv: "99594ffafc02b2bdbca85ad741e4239f"
server_key_version: "encryptionkey@0"
I20260812 21:03:51.256685 19851 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 21:03:51.257437 19873 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:03:51.257598 19851 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 21:03:51.257689 19851 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568607354921-12282-0/minicluster-data/ts-0/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568607354921-12282-0/minicluster-data/ts-0/wal
uuid: "94daf355993f4a3dbb64c14e00465548"
format_stamp: "Formatted at 2026-08-12 21:03:51 on dist-test-slave-vpvm"
server_key: "7d0b6d8f1570dad53003bb939916b159"
server_key_iv: "99594ffafc02b2bdbca85ad741e4239f"
server_key_version: "encryptionkey@0"
I20260812 21:03:51.257750 19851 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568607354921-12282-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568607354921-12282-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568607354921-12282-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:03:51.290499 19851 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:03:51.292775 19851 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:03:51.292922 19851 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:03:51.293160 19851 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:03:51.293489 19851 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:03:51.293529 19851 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:03:51.293560 19851 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:03:51.293581 19851 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:03:51.299283 19851 rpc_server.cc:307] RPC server started. Bound to: 127.11.254.129:36431
I20260812 21:03:51.299332 20052 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.254.129:36431 every 8 connection(s)
I20260812 21:03:51.299606 19851 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568607354921-12282-0/minicluster-data/ts-0/data/info.pb
I20260812 21:03:51.301872 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 19851
I20260812 21:03:51.301996 12282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568607354921-12282-0/minicluster-data/ts-0/wal/instance
I20260812 21:03:51.302218 12282 external_mini_cluster.cc:1428] Setting key 572147a53f5af0ff1a2991b9b33c9b73
Aug 12 21:03:51 dist-test-slave-vpvm krb5kdc[19654](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568631, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.129@KRBTEST.COM for kudu/127.11.254.190@KRBTEST.COM
I20260812 21:03:51.309343 20062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:51.300278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.129:37173 (local address 127.11.254.190:45239)
0812 21:03:51.300391 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0812 21:03:51.300393 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:03:51.300542 (+ 149us) server_negotiation.cc:407] Connection header received
0812 21:03:51.301011 (+ 469us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:03:51.301012 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:03:51.301042 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:03:51.301122 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:03:51.302190 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:51.302920 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:51.303996 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:51.304270 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:51.304341 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:51.306394 (+ 2053us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:03:51.306396 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:03:51.306396 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:03:51.306404 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:03:51.308026 (+ 1622us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:51.308411 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:51.308412 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:51.308412 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:51.308492 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:51.308703 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:51.308704 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:51.308704 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:51.308848 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:03:51.308920 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0812 21:03:51.309146 (+ 226us) server_negotiation.cc:299] Negotiation successful
0812 21:03:51.309236 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89,"thread_start_us":59,"threads_started":1}
I20260812 21:03:51.309403 20060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:51.300240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:45239 (local address 127.11.254.129:37173)
0812 21:03:51.300405 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0812 21:03:51.300450 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 21:03:51.300931 (+ 481us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:03:51.301186 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:03:51.301190 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:03:51.301452 (+ 262us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:03:51.302083 (+ 631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:51.302090 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:51.303015 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:51.303016 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:51.303903 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:51.303906 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:51.304349 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:51.304350 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:51.304461 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:51.304941 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:03:51.304950 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:03:51.306307 (+ 1357us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:03:51.308120 (+ 1813us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:03:51.308127 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:03:51.308129 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:03:51.308346 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:03:51.308561 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:03:51.308561 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:03:51.308562 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:03:51.308650 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:03:51.308905 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:03:51.308908 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:03:51.309028 (+ 120us) client_negotiation.cc:769] Sending connection context
0812 21:03:51.309106 (+ 78us) client_negotiation.cc:240] Negotiation successful
0812 21:03:51.309230 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":74,"threads_started":1}
I20260812 21:03:51.309973 20056 heartbeater.cc:347] Connected to a master server at 127.11.254.190:45239
I20260812 21:03:51.310086 20056 heartbeater.cc:464] Registering TS with master...
I20260812 21:03:51.310262 20056 heartbeater.cc:511] Master 127.11.254.190:45239 requested a full tablet report, sending...
I20260812 21:03:51.310803 19735 ts_manager.cc:194] Registered new tserver with Master: 94daf355993f4a3dbb64c14e00465548 (127.11.254.129:36431)
WARNING: no policy specified for kudu/127.11.254.130@KRBTEST.COM; defaulting to no policy
I20260812 21:03:51.311401 19735 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.254.129@KRBTEST.COM'} at 127.11.254.129:37173
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.254.130@KRBTEST.COM" created.
I20260812 21:03:52.312337 20056 heartbeater.cc:503] Master 127.11.254.190:45239 was elected leader, sending a full tablet report...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.254.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.130.keytab.
Entry for principal kudu/127.11.254.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.130.keytab.
W20260812 21:03:52.316138 12282 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.11.254.130: real 1.013s user 0.000s sys 0.004s
I20260812 21:03:52.316622 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568607354921-12282-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568607354921-12282-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568607354921-12282-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568607354921-12282-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.254.130:0
--local_ip_for_outbound_sockets=127.11.254.130
--webserver_interface=127.11.254.130
--webserver_port=0
--tserver_master_addrs=127.11.254.190:45239
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.130.keytab
--principal=kudu/127.11.254.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.254.148:38427
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568607354921-12282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568607354921-12282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568607354921-12282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:03:52.398849 20068 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:03:52.399088 20068 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:03:52.399123 20068 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:03:52.400321 20068 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:03:52.400362 20068 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:03:52.400398 20068 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.254.130
I20260812 21:03:52.401986 20068 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.254.148:38427
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568607354921-12282-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568607354921-12282-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.130.keytab
--principal=kudu/127.11.254.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.254.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568607354921-12282-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.254.130
--webserver_port=0
--tserver_master_addrs=127.11.254.190:45239
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20068
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.254.130
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568607354921-12282-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:03:52.402240 20068 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:03:52.402464 20068 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 21:03:52.407931 20068 server_base.cc:1034] running on GCE node
W20260812 21:03:52.407891 20082 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:03:52.407898 20086 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:03:52.407898 20079 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:03:52.408284 20068 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:03:52.408533 20068 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:03:52.409691 20068 hybrid_clock.cc:648] HybridClock initialized: now 1786568632409674 us; error 38 us; skew 500 ppm
Aug 12 21:03:52 dist-test-slave-vpvm krb5kdc[19654](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568632, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:03:52.412551 20068 init.cc:377] Logged in from keytab as kudu/127.11.254.130@KRBTEST.COM (short username kudu)
I20260812 21:03:52.412915 20068 fs_manager.cc:359] Metadata directory not provided
I20260812 21:03:52.412974 20068 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:03:52.413110 20068 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:03:52.414318 20068 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568607354921-12282-0/minicluster-data/ts-1/data/instance:
uuid: "8e7918945fe6426db5a4cc6c88716c13"
format_stamp: "Formatted at 2026-08-12 21:03:52 on dist-test-slave-vpvm"
server_key: "bc02c4798e591b75a684b4a03dae8392"
server_key_iv: "03cc59d88c477d5ae999db1b28fc1d38"
server_key_version: "encryptionkey@0"
I20260812 21:03:52.414706 20068 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568607354921-12282-0/minicluster-data/ts-1/wal/instance:
uuid: "8e7918945fe6426db5a4cc6c88716c13"
format_stamp: "Formatted at 2026-08-12 21:03:52 on dist-test-slave-vpvm"
server_key: "bc02c4798e591b75a684b4a03dae8392"
server_key_iv: "03cc59d88c477d5ae999db1b28fc1d38"
server_key_version: "encryptionkey@0"
I20260812 21:03:52.416067 20068 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 21:03:52.416924 20094 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:03:52.417101 20068 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 21:03:52.417196 20068 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568607354921-12282-0/minicluster-data/ts-1/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568607354921-12282-0/minicluster-data/ts-1/wal
uuid: "8e7918945fe6426db5a4cc6c88716c13"
format_stamp: "Formatted at 2026-08-12 21:03:52 on dist-test-slave-vpvm"
server_key: "bc02c4798e591b75a684b4a03dae8392"
server_key_iv: "03cc59d88c477d5ae999db1b28fc1d38"
server_key_version: "encryptionkey@0"
I20260812 21:03:52.417264 20068 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568607354921-12282-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568607354921-12282-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568607354921-12282-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:03:52.428138 20068 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:03:52.430327 20068 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:03:52.430450 20068 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:03:52.430644 20068 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:03:52.430913 20068 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:03:52.430944 20068 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:03:52.430966 20068 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:03:52.430977 20068 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:03:52.436641 20068 rpc_server.cc:307] RPC server started. Bound to: 127.11.254.130:43995
I20260812 21:03:52.436700 20305 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.254.130:43995 every 8 connection(s)
I20260812 21:03:52.436965 20068 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568607354921-12282-0/minicluster-data/ts-1/data/info.pb
I20260812 21:03:52.442307 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 20068
I20260812 21:03:52.442407 12282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568607354921-12282-0/minicluster-data/ts-1/wal/instance
I20260812 21:03:52.442584 12282 external_mini_cluster.cc:1428] Setting key 9628ee53a473315f8cae9e8a1784a9b8
Aug 12 21:03:52 dist-test-slave-vpvm krb5kdc[19654](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568632, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.130@KRBTEST.COM for kudu/127.11.254.190@KRBTEST.COM
I20260812 21:03:52.447033 20315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:52.437711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.130:37971 (local address 127.11.254.190:45239)
0812 21:03:52.438068 (+ 357us) server_negotiation.cc:206] Beginning negotiation
0812 21:03:52.438070 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:03:52.438082 (+ 12us) server_negotiation.cc:407] Connection header received
0812 21:03:52.438525 (+ 443us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:03:52.438526 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:03:52.438543 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:03:52.438593 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:03:52.439635 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:52.440175 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:52.441237 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:52.441451 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:52.441510 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:52.443975 (+ 2465us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:03:52.443976 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:03:52.443976 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:03:52.443983 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:03:52.445526 (+ 1543us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:52.445916 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:52.445917 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:52.445918 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:52.446013 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:52.446326 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:52.446328 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:52.446328 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:52.446474 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:03:52.446540 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0812 21:03:52.446831 (+ 291us) server_negotiation.cc:299] Negotiation successful
0812 21:03:52.446926 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":333,"thread_start_us":66,"threads_started":1}
I20260812 21:03:52.447129 20310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:52.437666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:45239 (local address 127.11.254.130:37971)
0812 21:03:52.437838 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0812 21:03:52.437879 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 21:03:52.438425 (+ 546us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:03:52.438676 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:03:52.438680 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:03:52.438944 (+ 264us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:03:52.439475 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:52.439481 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:52.440267 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:52.440268 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:52.441153 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:52.441156 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:52.441530 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:52.441531 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:52.441649 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:52.442174 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:03:52.442182 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:03:52.443884 (+ 1702us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:03:52.445614 (+ 1730us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:03:52.445620 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:03:52.445622 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:03:52.445845 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:03:52.446133 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:03:52.446134 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:03:52.446135 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:03:52.446254 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:03:52.446559 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:03:52.446565 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:03:52.446702 (+ 137us) client_negotiation.cc:769] Sending connection context
0812 21:03:52.446793 (+ 91us) client_negotiation.cc:240] Negotiation successful
0812 21:03:52.446934 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":75,"threads_started":1}
I20260812 21:03:52.447655 20306 heartbeater.cc:347] Connected to a master server at 127.11.254.190:45239
I20260812 21:03:52.447750 20306 heartbeater.cc:464] Registering TS with master...
I20260812 21:03:52.447899 20306 heartbeater.cc:511] Master 127.11.254.190:45239 requested a full tablet report, sending...
I20260812 21:03:52.448330 19735 ts_manager.cc:194] Registered new tserver with Master: 8e7918945fe6426db5a4cc6c88716c13 (127.11.254.130:43995)
I20260812 21:03:52.448827 19735 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.254.130@KRBTEST.COM'} at 127.11.254.130:37971
WARNING: no policy specified for kudu/127.11.254.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.254.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.254.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.131.keytab.
Entry for principal kudu/127.11.254.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.131.keytab.
I20260812 21:03:52.472321 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568607354921-12282-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568607354921-12282-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568607354921-12282-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568607354921-12282-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.254.131:0
--local_ip_for_outbound_sockets=127.11.254.131
--webserver_interface=127.11.254.131
--webserver_port=0
--tserver_master_addrs=127.11.254.190:45239
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.131.keytab
--principal=kudu/127.11.254.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.254.148:38427
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568607354921-12282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568607354921-12282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568607354921-12282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:03:52.557865 20322 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:03:52.558122 20322 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:03:52.558148 20322 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:03:52.559366 20322 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:03:52.559427 20322 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:03:52.559455 20322 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.254.131
I20260812 21:03:52.560993 20322 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.254.148:38427
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568607354921-12282-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568607354921-12282-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.131.keytab
--principal=kudu/127.11.254.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.254.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568607354921-12282-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.254.131
--webserver_port=0
--tserver_master_addrs=127.11.254.190:45239
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20322
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.254.131
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568607354921-12282-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:03:52.561205 20322 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:03:52.561416 20322 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:03:52.566509 20331 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:03:52.566519 20329 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:03:52.566509 20333 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:03:52.566758 20322 server_base.cc:1034] running on GCE node
I20260812 21:03:52.566953 20322 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:03:52.567160 20322 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:03:52.568328 20322 hybrid_clock.cc:648] HybridClock initialized: now 1786568632568313 us; error 26 us; skew 500 ppm
Aug 12 21:03:52 dist-test-slave-vpvm krb5kdc[19654](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568632, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:03:52.570873 20322 init.cc:377] Logged in from keytab as kudu/127.11.254.131@KRBTEST.COM (short username kudu)
I20260812 21:03:52.571175 20322 fs_manager.cc:359] Metadata directory not provided
I20260812 21:03:52.571218 20322 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:03:52.571314 20322 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:03:52.572368 20322 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568607354921-12282-0/minicluster-data/ts-2/data/instance:
uuid: "2a3ebaf632ea40fc9aa7a5ec7254a432"
format_stamp: "Formatted at 2026-08-12 21:03:52 on dist-test-slave-vpvm"
server_key: "d125b8ae3e9b19140a2e116b6cb91a67"
server_key_iv: "de71c9fc55704e384847073bd587d79d"
server_key_version: "encryptionkey@0"
I20260812 21:03:52.572687 20322 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568607354921-12282-0/minicluster-data/ts-2/wal/instance:
uuid: "2a3ebaf632ea40fc9aa7a5ec7254a432"
format_stamp: "Formatted at 2026-08-12 21:03:52 on dist-test-slave-vpvm"
server_key: "d125b8ae3e9b19140a2e116b6cb91a67"
server_key_iv: "de71c9fc55704e384847073bd587d79d"
server_key_version: "encryptionkey@0"
I20260812 21:03:52.573879 20322 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 21:03:52.574612 20342 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:03:52.574769 20322 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 21:03:52.574854 20322 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568607354921-12282-0/minicluster-data/ts-2/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568607354921-12282-0/minicluster-data/ts-2/wal
uuid: "2a3ebaf632ea40fc9aa7a5ec7254a432"
format_stamp: "Formatted at 2026-08-12 21:03:52 on dist-test-slave-vpvm"
server_key: "d125b8ae3e9b19140a2e116b6cb91a67"
server_key_iv: "de71c9fc55704e384847073bd587d79d"
server_key_version: "encryptionkey@0"
I20260812 21:03:52.574909 20322 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568607354921-12282-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568607354921-12282-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568607354921-12282-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:03:52.591897 20322 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:03:52.594062 20322 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:03:52.594188 20322 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:03:52.594395 20322 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:03:52.594674 20322 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:03:52.594704 20322 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:03:52.594726 20322 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:03:52.594743 20322 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:03:52.600207 20322 rpc_server.cc:307] RPC server started. Bound to: 127.11.254.131:45041
I20260812 21:03:52.600278 20521 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.254.131:45041 every 8 connection(s)
I20260812 21:03:52.600512 20322 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568607354921-12282-0/minicluster-data/ts-2/data/info.pb
Aug 12 21:03:52 dist-test-slave-vpvm krb5kdc[19654](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568632, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.131@KRBTEST.COM for kudu/127.11.254.190@KRBTEST.COM
I20260812 21:03:52.608642 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 20322
I20260812 21:03:52.608748 12282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568607354921-12282-0/minicluster-data/ts-2/wal/instance
I20260812 21:03:52.608927 12282 external_mini_cluster.cc:1428] Setting key fb0f928414b1333e20043b414693304d
I20260812 21:03:52.609447 20315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:52.601227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.131:39427 (local address 127.11.254.190:45239)
0812 21:03:52.601318 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0812 21:03:52.601320 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:03:52.601486 (+ 166us) server_negotiation.cc:407] Connection header received
0812 21:03:52.601969 (+ 483us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:03:52.601971 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:03:52.601981 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:03:52.602023 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:03:52.602954 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:52.603481 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:52.604514 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:52.604729 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:52.604784 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:52.606843 (+ 2059us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:03:52.606844 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:03:52.606845 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:03:52.606851 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:03:52.608126 (+ 1275us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:52.608499 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:52.608500 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:52.608500 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:52.608554 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:52.608764 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:52.608765 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:52.608765 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:52.608898 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:03:52.608956 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0812 21:03:52.609257 (+ 301us) server_negotiation.cc:299] Negotiation successful
0812 21:03:52.609357 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260812 21:03:52.609606 20527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:52.601188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:45239 (local address 127.11.254.131:39427)
0812 21:03:52.601366 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0812 21:03:52.601404 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 21:03:52.601879 (+ 475us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:03:52.602091 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:03:52.602095 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:03:52.602358 (+ 263us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:03:52.602882 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:52.602888 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:52.603606 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:52.603607 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:52.604439 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:52.604442 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:52.604793 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:52.604793 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:52.604889 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:52.605336 (+ 447us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:03:52.605344 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:03:52.606750 (+ 1406us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:03:52.608202 (+ 1452us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:03:52.608209 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:03:52.608211 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:03:52.608422 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:03:52.608608 (+ 186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:03:52.608609 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:03:52.608609 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:03:52.608711 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:03:52.608954 (+ 243us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:03:52.608960 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:03:52.609126 (+ 166us) client_negotiation.cc:769] Sending connection context
0812 21:03:52.609232 (+ 106us) client_negotiation.cc:240] Negotiation successful
0812 21:03:52.609385 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":96,"threads_started":1}
I20260812 21:03:52.610054 20524 heartbeater.cc:347] Connected to a master server at 127.11.254.190:45239
I20260812 21:03:52.610144 20524 heartbeater.cc:464] Registering TS with master...
I20260812 21:03:52.610289 20524 heartbeater.cc:511] Master 127.11.254.190:45239 requested a full tablet report, sending...
I20260812 21:03:52.610628 19735 ts_manager.cc:194] Registered new tserver with Master: 2a3ebaf632ea40fc9aa7a5ec7254a432 (127.11.254.131:45041)
I20260812 21:03:52.610953 19735 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.254.131@KRBTEST.COM'} at 127.11.254.131:39427
I20260812 21:03:52.621068 12282 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 21:03:52.628563 20315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:52.622467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33380 (local address 127.11.254.190:45239)
0812 21:03:52.622543 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0812 21:03:52.622545 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:03:52.622696 (+ 151us) server_negotiation.cc:407] Connection header received
0812 21:03:52.622745 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:03:52.622746 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:03:52.622756 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:03:52.622802 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:03:52.623435 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:52.623975 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:52.624723 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:52.624940 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:52.624999 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:52.626241 (+ 1242us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:03:52.626242 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:03:52.626242 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:03:52.626249 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:03:52.627600 (+ 1351us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:52.627914 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:52.627914 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:52.627915 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:52.627981 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:52.628119 (+ 138us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:52.628119 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:52.628120 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:52.628267 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:03:52.628340 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 21:03:52.628414 (+ 74us) server_negotiation.cc:299] Negotiation successful
0812 21:03:52.628491 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260812 21:03:52.630227 19735 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33380:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 21:03:52.631832 19735 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 21:03:52.639132 20552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:52.635067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51442 (local address 127.11.254.129:36431)
0812 21:03:52.635212 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0812 21:03:52.635217 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 21:03:52.635415 (+ 198us) server_negotiation.cc:407] Connection header received
0812 21:03:52.635686 (+ 271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:03:52.635689 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:03:52.635705 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:03:52.635757 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:03:52.635990 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:52.637064 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:52.638245 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:52.638734 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:52.638801 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:52.638861 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0812 21:03:52.639001 (+ 140us) server_negotiation.cc:299] Negotiation successful
0812 21:03:52.639045 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":53,"threads_started":1}
I20260812 21:03:52.639124 20546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:52.634866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.129:36431 (local address 127.0.0.1:51442)
0812 21:03:52.635296 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0812 21:03:52.635341 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 21:03:52.635585 (+ 244us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:03:52.635849 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:03:52.635850 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:03:52.635852 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:03:52.635908 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:52.635910 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:52.637159 (+ 1249us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:52.637160 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:52.638165 (+ 1005us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:52.638168 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:52.638811 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:52.638812 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:52.638891 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:52.638912 (+ 21us) client_negotiation.cc:769] Sending connection context
0812 21:03:52.638979 (+ 67us) client_negotiation.cc:240] Negotiation successful
0812 21:03:52.639037 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":395,"mutex_wait_us":1,"thread_start_us":102,"threads_started":1}
I20260812 21:03:52.639253 20543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:52.635115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.131:45041 (local address 127.0.0.1:32774)
0812 21:03:52.635334 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0812 21:03:52.635344 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 21:03:52.635421 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:03:52.635737 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:03:52.635738 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:03:52.635740 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:03:52.635798 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:52.635800 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:52.637199 (+ 1399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:52.637200 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:52.638163 (+ 963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:52.638166 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:52.639016 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:52.639017 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:52.639077 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:52.639082 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 21:03:52.639135 (+ 53us) client_negotiation.cc:240] Negotiation successful
0812 21:03:52.639176 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":184,"mutex_wait_us":3,"thread_start_us":68,"threads_started":1}
I20260812 21:03:52.639267 20554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:52.635221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32774 (local address 127.11.254.131:45041)
0812 21:03:52.635586 (+ 365us) server_negotiation.cc:206] Beginning negotiation
0812 21:03:52.635591 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 21:03:52.635599 (+ 8us) server_negotiation.cc:407] Connection header received
0812 21:03:52.635635 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:03:52.635637 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:03:52.635650 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:03:52.635874 (+ 224us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:03:52.635962 (+ 88us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:52.637088 (+ 1126us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:52.638255 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:52.638942 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:52.639004 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:52.639054 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0812 21:03:52.639149 (+ 95us) server_negotiation.cc:299] Negotiation successful
0812 21:03:52.639191 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":327,"thread_start_us":72,"threads_started":1}
I20260812 21:03:52.639575 20553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:52.634866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.130:43995 (local address 127.0.0.1:49946)
0812 21:03:52.635296 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0812 21:03:52.635341 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 21:03:52.635421 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:03:52.635708 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:03:52.635711 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:03:52.635713 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:03:52.635797 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:52.635800 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:52.636737 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:52.636738 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:52.637863 (+ 1125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:52.637866 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:52.639338 (+ 1472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:52.639339 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:52.639411 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:52.639415 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 21:03:52.639466 (+ 51us) client_negotiation.cc:240] Negotiation successful
0812 21:03:52.639508 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":382,"thread_start_us":100,"threads_started":1}
I20260812 21:03:52.639612 20548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:52.635000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49946 (local address 127.11.254.130:43995)
0812 21:03:52.635213 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0812 21:03:52.635217 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 21:03:52.635455 (+ 238us) server_negotiation.cc:407] Connection header received
0812 21:03:52.635507 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:03:52.635510 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:03:52.635526 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:03:52.635598 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:03:52.635866 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:52.636638 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:52.637976 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:52.638728 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:52.638805 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:52.638860 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0812 21:03:52.639489 (+ 629us) server_negotiation.cc:299] Negotiation successful
0812 21:03:52.639535 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":188,"thread_start_us":94,"threads_started":1}
I20260812 21:03:52.640027 20420 tablet_service.cc:1467] Processing CreateTablet for tablet 00f0c2d41b7c4f07b700213fd8587a9d (DEFAULT_TABLE table=test-table [id=36d057c148e64bf89ac4c3175e96a87f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:03:52.640231 19951 tablet_service.cc:1467] Processing CreateTablet for tablet 00f0c2d41b7c4f07b700213fd8587a9d (DEFAULT_TABLE table=test-table [id=36d057c148e64bf89ac4c3175e96a87f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:03:52.640374 20420 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00f0c2d41b7c4f07b700213fd8587a9d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:03:52.640513 19951 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00f0c2d41b7c4f07b700213fd8587a9d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:03:52.642670 20185 tablet_service.cc:1467] Processing CreateTablet for tablet 00f0c2d41b7c4f07b700213fd8587a9d (DEFAULT_TABLE table=test-table [id=36d057c148e64bf89ac4c3175e96a87f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:03:52.643049 20185 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00f0c2d41b7c4f07b700213fd8587a9d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:03:52.643275 20557 tablet_bootstrap.cc:492] T 00f0c2d41b7c4f07b700213fd8587a9d P 94daf355993f4a3dbb64c14e00465548: Bootstrap starting.
I20260812 21:03:52.643585 20559 tablet_bootstrap.cc:492] T 00f0c2d41b7c4f07b700213fd8587a9d P 2a3ebaf632ea40fc9aa7a5ec7254a432: Bootstrap starting.
I20260812 21:03:52.643862 20557 tablet_bootstrap.cc:654] T 00f0c2d41b7c4f07b700213fd8587a9d P 94daf355993f4a3dbb64c14e00465548: Neither blocks nor log segments found. Creating new log.
I20260812 21:03:52.644152 20557 log.cc:824] T 00f0c2d41b7c4f07b700213fd8587a9d P 94daf355993f4a3dbb64c14e00465548: Log is configured to *not* fsync() on all Append() calls
I20260812 21:03:52.644213 20559 tablet_bootstrap.cc:654] T 00f0c2d41b7c4f07b700213fd8587a9d P 2a3ebaf632ea40fc9aa7a5ec7254a432: Neither blocks nor log segments found. Creating new log.
I20260812 21:03:52.644901 20559 log.cc:824] T 00f0c2d41b7c4f07b700213fd8587a9d P 2a3ebaf632ea40fc9aa7a5ec7254a432: Log is configured to *not* fsync() on all Append() calls
I20260812 21:03:52.645016 20557 tablet_bootstrap.cc:492] T 00f0c2d41b7c4f07b700213fd8587a9d P 94daf355993f4a3dbb64c14e00465548: No bootstrap required, opened a new log
I20260812 21:03:52.645085 20557 ts_tablet_manager.cc:1397] T 00f0c2d41b7c4f07b700213fd8587a9d P 94daf355993f4a3dbb64c14e00465548: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 21:03:52.645753 20559 tablet_bootstrap.cc:492] T 00f0c2d41b7c4f07b700213fd8587a9d P 2a3ebaf632ea40fc9aa7a5ec7254a432: No bootstrap required, opened a new log
I20260812 21:03:52.645829 20559 ts_tablet_manager.cc:1397] T 00f0c2d41b7c4f07b700213fd8587a9d P 2a3ebaf632ea40fc9aa7a5ec7254a432: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 21:03:52.646401 20565 tablet_bootstrap.cc:492] T 00f0c2d41b7c4f07b700213fd8587a9d P 8e7918945fe6426db5a4cc6c88716c13: Bootstrap starting.
I20260812 21:03:52.647116 20565 tablet_bootstrap.cc:654] T 00f0c2d41b7c4f07b700213fd8587a9d P 8e7918945fe6426db5a4cc6c88716c13: Neither blocks nor log segments found. Creating new log.
I20260812 21:03:52.647447 20559 raft_consensus.cc:348] T 00f0c2d41b7c4f07b700213fd8587a9d P 2a3ebaf632ea40fc9aa7a5ec7254a432 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94daf355993f4a3dbb64c14e00465548" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 36431 } } peers { permanent_uuid: "8e7918945fe6426db5a4cc6c88716c13" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 43995 } } peers { permanent_uuid: "2a3ebaf632ea40fc9aa7a5ec7254a432" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 45041 } }
I20260812 21:03:52.647449 20557 raft_consensus.cc:348] T 00f0c2d41b7c4f07b700213fd8587a9d P 94daf355993f4a3dbb64c14e00465548 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94daf355993f4a3dbb64c14e00465548" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 36431 } } peers { permanent_uuid: "8e7918945fe6426db5a4cc6c88716c13" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 43995 } } peers { permanent_uuid: "2a3ebaf632ea40fc9aa7a5ec7254a432" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 45041 } }
I20260812 21:03:52.647591 20559 raft_consensus.cc:374] T 00f0c2d41b7c4f07b700213fd8587a9d P 2a3ebaf632ea40fc9aa7a5ec7254a432 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:03:52.647609 20557 raft_consensus.cc:374] T 00f0c2d41b7c4f07b700213fd8587a9d P 94daf355993f4a3dbb64c14e00465548 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:03:52.647624 20559 raft_consensus.cc:729] T 00f0c2d41b7c4f07b700213fd8587a9d P 2a3ebaf632ea40fc9aa7a5ec7254a432 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a3ebaf632ea40fc9aa7a5ec7254a432, State: Initialized, Role: FOLLOWER
I20260812 21:03:52.647650 20557 raft_consensus.cc:729] T 00f0c2d41b7c4f07b700213fd8587a9d P 94daf355993f4a3dbb64c14e00465548 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94daf355993f4a3dbb64c14e00465548, State: Initialized, Role: FOLLOWER
I20260812 21:03:52.647684 20565 log.cc:824] T 00f0c2d41b7c4f07b700213fd8587a9d P 8e7918945fe6426db5a4cc6c88716c13: Log is configured to *not* fsync() on all Append() calls
I20260812 21:03:52.647760 20559 consensus_queue.cc:260] T 00f0c2d41b7c4f07b700213fd8587a9d P 2a3ebaf632ea40fc9aa7a5ec7254a432 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94daf355993f4a3dbb64c14e00465548" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 36431 } } peers { permanent_uuid: "8e7918945fe6426db5a4cc6c88716c13" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 43995 } } peers { permanent_uuid: "2a3ebaf632ea40fc9aa7a5ec7254a432" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 45041 } }
I20260812 21:03:52.647779 20557 consensus_queue.cc:260] T 00f0c2d41b7c4f07b700213fd8587a9d P 94daf355993f4a3dbb64c14e00465548 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94daf355993f4a3dbb64c14e00465548" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 36431 } } peers { permanent_uuid: "8e7918945fe6426db5a4cc6c88716c13" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 43995 } } peers { permanent_uuid: "2a3ebaf632ea40fc9aa7a5ec7254a432" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 45041 } }
I20260812 21:03:52.648037 20557 ts_tablet_manager.cc:1428] T 00f0c2d41b7c4f07b700213fd8587a9d P 94daf355993f4a3dbb64c14e00465548: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 21:03:52.648037 20559 ts_tablet_manager.cc:1428] T 00f0c2d41b7c4f07b700213fd8587a9d P 2a3ebaf632ea40fc9aa7a5ec7254a432: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260812 21:03:52.648061 20524 heartbeater.cc:503] Master 127.11.254.190:45239 was elected leader, sending a full tablet report...
I20260812 21:03:52.648870 20565 tablet_bootstrap.cc:492] T 00f0c2d41b7c4f07b700213fd8587a9d P 8e7918945fe6426db5a4cc6c88716c13: No bootstrap required, opened a new log
I20260812 21:03:52.648937 20565 ts_tablet_manager.cc:1397] T 00f0c2d41b7c4f07b700213fd8587a9d P 8e7918945fe6426db5a4cc6c88716c13: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 21:03:52.650561 20565 raft_consensus.cc:348] T 00f0c2d41b7c4f07b700213fd8587a9d P 8e7918945fe6426db5a4cc6c88716c13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94daf355993f4a3dbb64c14e00465548" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 36431 } } peers { permanent_uuid: "8e7918945fe6426db5a4cc6c88716c13" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 43995 } } peers { permanent_uuid: "2a3ebaf632ea40fc9aa7a5ec7254a432" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 45041 } }
I20260812 21:03:52.650701 20565 raft_consensus.cc:374] T 00f0c2d41b7c4f07b700213fd8587a9d P 8e7918945fe6426db5a4cc6c88716c13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:03:52.650733 20565 raft_consensus.cc:729] T 00f0c2d41b7c4f07b700213fd8587a9d P 8e7918945fe6426db5a4cc6c88716c13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e7918945fe6426db5a4cc6c88716c13, State: Initialized, Role: FOLLOWER
I20260812 21:03:52.650846 20565 consensus_queue.cc:260] T 00f0c2d41b7c4f07b700213fd8587a9d P 8e7918945fe6426db5a4cc6c88716c13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94daf355993f4a3dbb64c14e00465548" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 36431 } } peers { permanent_uuid: "8e7918945fe6426db5a4cc6c88716c13" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 43995 } } peers { permanent_uuid: "2a3ebaf632ea40fc9aa7a5ec7254a432" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 45041 } }
I20260812 21:03:52.651093 20565 ts_tablet_manager.cc:1428] T 00f0c2d41b7c4f07b700213fd8587a9d P 8e7918945fe6426db5a4cc6c88716c13: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260812 21:03:52.651139 20306 heartbeater.cc:503] Master 127.11.254.190:45239 was elected leader, sending a full tablet report...
W20260812 21:03:52.687582 20308 tablet.cc:2369] T 00f0c2d41b7c4f07b700213fd8587a9d P 8e7918945fe6426db5a4cc6c88716c13: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 21:03:52.801014 20059 tablet.cc:2369] T 00f0c2d41b7c4f07b700213fd8587a9d P 94daf355993f4a3dbb64c14e00465548: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 21:03:52.850636 20573 raft_consensus.cc:482] T 00f0c2d41b7c4f07b700213fd8587a9d P 8e7918945fe6426db5a4cc6c88716c13 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 21:03:52.850764 20573 raft_consensus.cc:504] T 00f0c2d41b7c4f07b700213fd8587a9d P 8e7918945fe6426db5a4cc6c88716c13 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94daf355993f4a3dbb64c14e00465548" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 36431 } } peers { permanent_uuid: "8e7918945fe6426db5a4cc6c88716c13" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 43995 } } peers { permanent_uuid: "2a3ebaf632ea40fc9aa7a5ec7254a432" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 45041 } }
W20260812 21:03:52.851058 20525 tablet.cc:2369] T 00f0c2d41b7c4f07b700213fd8587a9d P 2a3ebaf632ea40fc9aa7a5ec7254a432: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 21:03:52.851150 20573 leader_election.cc:288] T 00f0c2d41b7c4f07b700213fd8587a9d P 8e7918945fe6426db5a4cc6c88716c13 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 94daf355993f4a3dbb64c14e00465548 (127.11.254.129:36431), 2a3ebaf632ea40fc9aa7a5ec7254a432 (127.11.254.131:45041)
I20260812 21:03:52.853905 20552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:52.851474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.130:38401 (local address 127.11.254.129:36431)
0812 21:03:52.851528 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0812 21:03:52.851530 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:03:52.851540 (+ 10us) server_negotiation.cc:407] Connection header received
0812 21:03:52.851573 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:03:52.851573 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:03:52.851582 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:03:52.851659 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:03:52.851809 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:52.852287 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:52.853211 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:52.853595 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:52.853660 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:52.853668 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 21:03:52.853800 (+ 132us) server_negotiation.cc:299] Negotiation successful
0812 21:03:52.853837 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260812 21:03:52.853910 20310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:52.851344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.129:36431 (local address 127.11.254.130:38401)
0812 21:03:52.851403 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0812 21:03:52.851414 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 21:03:52.851465 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:03:52.851671 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:03:52.851673 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:03:52.851676 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:03:52.851748 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:52.851750 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:52.852363 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:52.852364 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:52.853126 (+ 762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:52.853129 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:52.853665 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:52.853666 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:52.853738 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:52.853741 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 21:03:52.853787 (+ 46us) client_negotiation.cc:240] Negotiation successful
0812 21:03:52.853826 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260812 21:03:52.853986 20554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:52.851538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.130:45507 (local address 127.11.254.131:45041)
0812 21:03:52.851606 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0812 21:03:52.851608 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:03:52.851694 (+ 86us) server_negotiation.cc:407] Connection header received
0812 21:03:52.851742 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:03:52.851743 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:03:52.851752 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:03:52.851799 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:03:52.851934 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:52.852386 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:52.853278 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:52.853663 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:52.853724 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:52.853732 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 21:03:52.853868 (+ 136us) server_negotiation.cc:299] Negotiation successful
0812 21:03:52.853902 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260812 21:03:52.854044 20574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:52.851442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.131:45041 (local address 127.11.254.130:45507)
0812 21:03:52.851629 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0812 21:03:52.851637 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0812 21:03:52.851690 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:03:52.851805 (+ 115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:03:52.851806 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:03:52.851807 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:03:52.851875 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:52.851877 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:52.852460 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:52.852461 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:52.853191 (+ 730us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:52.853193 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:52.853735 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:52.853736 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:52.853806 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:52.853809 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 21:03:52.853854 (+ 45us) client_negotiation.cc:240] Negotiation successful
0812 21:03:52.853893 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":118,"threads_started":1}
I20260812 21:03:52.854350 20450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00f0c2d41b7c4f07b700213fd8587a9d" candidate_uuid: "8e7918945fe6426db5a4cc6c88716c13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a3ebaf632ea40fc9aa7a5ec7254a432" is_pre_election: true
I20260812 21:03:52.854360 19991 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00f0c2d41b7c4f07b700213fd8587a9d" candidate_uuid: "8e7918945fe6426db5a4cc6c88716c13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94daf355993f4a3dbb64c14e00465548" is_pre_election: true
I20260812 21:03:52.854496 20450 raft_consensus.cc:2454] T 00f0c2d41b7c4f07b700213fd8587a9d P 2a3ebaf632ea40fc9aa7a5ec7254a432 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e7918945fe6426db5a4cc6c88716c13 in term 0.
I20260812 21:03:52.854508 19991 raft_consensus.cc:2454] T 00f0c2d41b7c4f07b700213fd8587a9d P 94daf355993f4a3dbb64c14e00465548 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e7918945fe6426db5a4cc6c88716c13 in term 0.
I20260812 21:03:52.854707 20098 leader_election.cc:302] T 00f0c2d41b7c4f07b700213fd8587a9d P 8e7918945fe6426db5a4cc6c88716c13 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a3ebaf632ea40fc9aa7a5ec7254a432, 8e7918945fe6426db5a4cc6c88716c13; no voters:
I20260812 21:03:52.854825 20573 raft_consensus.cc:2781] T 00f0c2d41b7c4f07b700213fd8587a9d P 8e7918945fe6426db5a4cc6c88716c13 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 21:03:52.854882 20573 raft_consensus.cc:482] T 00f0c2d41b7c4f07b700213fd8587a9d P 8e7918945fe6426db5a4cc6c88716c13 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 21:03:52.854904 20573 raft_consensus.cc:3037] T 00f0c2d41b7c4f07b700213fd8587a9d P 8e7918945fe6426db5a4cc6c88716c13 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:03:52.855605 20573 raft_consensus.cc:504] T 00f0c2d41b7c4f07b700213fd8587a9d P 8e7918945fe6426db5a4cc6c88716c13 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94daf355993f4a3dbb64c14e00465548" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 36431 } } peers { permanent_uuid: "8e7918945fe6426db5a4cc6c88716c13" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 43995 } } peers { permanent_uuid: "2a3ebaf632ea40fc9aa7a5ec7254a432" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 45041 } }
I20260812 21:03:52.855715 20573 leader_election.cc:288] T 00f0c2d41b7c4f07b700213fd8587a9d P 8e7918945fe6426db5a4cc6c88716c13 [CANDIDATE]: Term 1 election: Requested vote from peers 94daf355993f4a3dbb64c14e00465548 (127.11.254.129:36431), 2a3ebaf632ea40fc9aa7a5ec7254a432 (127.11.254.131:45041)
I20260812 21:03:52.855875 19991 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00f0c2d41b7c4f07b700213fd8587a9d" candidate_uuid: "8e7918945fe6426db5a4cc6c88716c13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94daf355993f4a3dbb64c14e00465548"
I20260812 21:03:52.855916 20450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00f0c2d41b7c4f07b700213fd8587a9d" candidate_uuid: "8e7918945fe6426db5a4cc6c88716c13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a3ebaf632ea40fc9aa7a5ec7254a432"
I20260812 21:03:52.855974 20450 raft_consensus.cc:3037] T 00f0c2d41b7c4f07b700213fd8587a9d P 2a3ebaf632ea40fc9aa7a5ec7254a432 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:03:52.855965 19991 raft_consensus.cc:3037] T 00f0c2d41b7c4f07b700213fd8587a9d P 94daf355993f4a3dbb64c14e00465548 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:03:52.856635 20450 raft_consensus.cc:2454] T 00f0c2d41b7c4f07b700213fd8587a9d P 2a3ebaf632ea40fc9aa7a5ec7254a432 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e7918945fe6426db5a4cc6c88716c13 in term 1.
I20260812 21:03:52.856725 19991 raft_consensus.cc:2454] T 00f0c2d41b7c4f07b700213fd8587a9d P 94daf355993f4a3dbb64c14e00465548 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e7918945fe6426db5a4cc6c88716c13 in term 1.
I20260812 21:03:52.856787 20098 leader_election.cc:302] T 00f0c2d41b7c4f07b700213fd8587a9d P 8e7918945fe6426db5a4cc6c88716c13 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a3ebaf632ea40fc9aa7a5ec7254a432, 8e7918945fe6426db5a4cc6c88716c13; no voters:
I20260812 21:03:52.856906 20573 raft_consensus.cc:2781] T 00f0c2d41b7c4f07b700213fd8587a9d P 8e7918945fe6426db5a4cc6c88716c13 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:03:52.857072 20573 raft_consensus.cc:686] T 00f0c2d41b7c4f07b700213fd8587a9d P 8e7918945fe6426db5a4cc6c88716c13 [term 1 LEADER]: Becoming Leader. State: Replica: 8e7918945fe6426db5a4cc6c88716c13, State: Running, Role: LEADER
I20260812 21:03:52.857168 20573 consensus_queue.cc:237] T 00f0c2d41b7c4f07b700213fd8587a9d P 8e7918945fe6426db5a4cc6c88716c13 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94daf355993f4a3dbb64c14e00465548" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 36431 } } peers { permanent_uuid: "8e7918945fe6426db5a4cc6c88716c13" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 43995 } } peers { permanent_uuid: "2a3ebaf632ea40fc9aa7a5ec7254a432" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 45041 } }
I20260812 21:03:52.858090 19735 catalog_manager.cc:5495] T 00f0c2d41b7c4f07b700213fd8587a9d P 8e7918945fe6426db5a4cc6c88716c13 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8e7918945fe6426db5a4cc6c88716c13 (127.11.254.130). New cstate: current_term: 1 leader_uuid: "8e7918945fe6426db5a4cc6c88716c13" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94daf355993f4a3dbb64c14e00465548" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 36431 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8e7918945fe6426db5a4cc6c88716c13" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 43995 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2a3ebaf632ea40fc9aa7a5ec7254a432" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 45041 } health_report { overall_health: UNKNOWN } } }
I20260812 21:03:52.868553 20548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:52.866303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49948 (local address 127.11.254.130:43995)
0812 21:03:52.866353 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0812 21:03:52.866355 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:03:52.866365 (+ 10us) server_negotiation.cc:407] Connection header received
0812 21:03:52.866428 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:03:52.866429 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:03:52.866445 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:03:52.866499 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 21:03:52.866622 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:52.867081 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:52.867891 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:52.868077 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:52.868121 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:52.868304 (+ 183us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 21:03:52.868398 (+ 94us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 21:03:52.868452 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0812 21:03:52.868467 (+ 15us) server_negotiation.cc:299] Negotiation successful
0812 21:03:52.868483 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260812 21:03:52.870297 19991 raft_consensus.cc:1261] T 00f0c2d41b7c4f07b700213fd8587a9d P 94daf355993f4a3dbb64c14e00465548 [term 1 FOLLOWER]: Refusing update from remote peer 8e7918945fe6426db5a4cc6c88716c13: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 21:03:52.870304 20450 raft_consensus.cc:1261] T 00f0c2d41b7c4f07b700213fd8587a9d P 2a3ebaf632ea40fc9aa7a5ec7254a432 [term 1 FOLLOWER]: Refusing update from remote peer 8e7918945fe6426db5a4cc6c88716c13: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 21:03:52.870622 20575 consensus_queue.cc:1035] T 00f0c2d41b7c4f07b700213fd8587a9d P 8e7918945fe6426db5a4cc6c88716c13 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94daf355993f4a3dbb64c14e00465548" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 36431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 21:03:52.870759 20573 consensus_queue.cc:1035] T 00f0c2d41b7c4f07b700213fd8587a9d P 8e7918945fe6426db5a4cc6c88716c13 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a3ebaf632ea40fc9aa7a5ec7254a432" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 45041 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 21:03:52.873029 20579 mvcc.cc:205] Tried to move back new op lower bound from 7317785120234225664 to 7317785120183488512. Current Snapshot: MvccSnapshot[applied={T|T < 7317785120234225664}]
I20260812 21:03:52.873376 20582 mvcc.cc:205] Tried to move back new op lower bound from 7317785120234225664 to 7317785120183488512. Current Snapshot: MvccSnapshot[applied={T|T < 7317785120234225664}]
I20260812 21:03:52.875658 19735 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33380:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 21:03:52.875743 19735 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33380:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 21:03:52.876435 19735 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2c39843ab1224081ab700e6a3b53cb5a: Sending DeleteTablet for 3 replicas of tablet 00f0c2d41b7c4f07b700213fd8587a9d
I20260812 21:03:52.876783 20185 tablet_service.cc:1514] Processing DeleteTablet for tablet 00f0c2d41b7c4f07b700213fd8587a9d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:03:52 UTC) from {username='kudu', principal='kudu/127.11.254.190@KRBTEST.COM'} at 127.0.0.1:49946
I20260812 21:03:52.876825 20420 tablet_service.cc:1514] Processing DeleteTablet for tablet 00f0c2d41b7c4f07b700213fd8587a9d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:03:52 UTC) from {username='kudu', principal='kudu/127.11.254.190@KRBTEST.COM'} at 127.0.0.1:32774
I20260812 21:03:52.876993 20595 tablet_replica.cc:330] T 00f0c2d41b7c4f07b700213fd8587a9d P 8e7918945fe6426db5a4cc6c88716c13: stopping tablet replica
I20260812 21:03:52.877084 20595 raft_consensus.cc:2229] T 00f0c2d41b7c4f07b700213fd8587a9d P 8e7918945fe6426db5a4cc6c88716c13 [term 1 LEADER]: Raft consensus shutting down.
I20260812 21:03:52.877125 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 19851
I20260812 21:03:52.877204 20595 raft_consensus.cc:2258] T 00f0c2d41b7c4f07b700213fd8587a9d P 8e7918945fe6426db5a4cc6c88716c13 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 21:03:52.888561 20596 tablet_replica.cc:330] T 00f0c2d41b7c4f07b700213fd8587a9d P 2a3ebaf632ea40fc9aa7a5ec7254a432: stopping tablet replica
I20260812 21:03:52.888747 20596 raft_consensus.cc:2229] T 00f0c2d41b7c4f07b700213fd8587a9d P 2a3ebaf632ea40fc9aa7a5ec7254a432 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 21:03:52.888882 20596 raft_consensus.cc:2258] T 00f0c2d41b7c4f07b700213fd8587a9d P 2a3ebaf632ea40fc9aa7a5ec7254a432 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 21:03:52.889493 20595 ts_tablet_manager.cc:1905] T 00f0c2d41b7c4f07b700213fd8587a9d P 8e7918945fe6426db5a4cc6c88716c13: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 21:03:52.889493 20596 ts_tablet_manager.cc:1905] T 00f0c2d41b7c4f07b700213fd8587a9d P 2a3ebaf632ea40fc9aa7a5ec7254a432: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 21:03:52.891372 20596 ts_tablet_manager.cc:1918] T 00f0c2d41b7c4f07b700213fd8587a9d P 2a3ebaf632ea40fc9aa7a5ec7254a432: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 21:03:52.891387 20595 ts_tablet_manager.cc:1918] T 00f0c2d41b7c4f07b700213fd8587a9d P 8e7918945fe6426db5a4cc6c88716c13: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 21:03:52.891472 20595 log.cc:1196] T 00f0c2d41b7c4f07b700213fd8587a9d P 8e7918945fe6426db5a4cc6c88716c13: Deleting WAL directory at /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568607354921-12282-0/minicluster-data/ts-1/wal/wals/00f0c2d41b7c4f07b700213fd8587a9d
I20260812 21:03:52.891475 20596 log.cc:1196] T 00f0c2d41b7c4f07b700213fd8587a9d P 2a3ebaf632ea40fc9aa7a5ec7254a432: Deleting WAL directory at /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786568607354921-12282-0/minicluster-data/ts-2/wal/wals/00f0c2d41b7c4f07b700213fd8587a9d
I20260812 21:03:52.891757 20595 ts_tablet_manager.cc:1939] T 00f0c2d41b7c4f07b700213fd8587a9d P 8e7918945fe6426db5a4cc6c88716c13: Deleting consensus metadata
I20260812 21:03:52.891788 20596 ts_tablet_manager.cc:1939] T 00f0c2d41b7c4f07b700213fd8587a9d P 2a3ebaf632ea40fc9aa7a5ec7254a432: Deleting consensus metadata
I20260812 21:03:52.892347 19709 catalog_manager.cc:4841] TS 8e7918945fe6426db5a4cc6c88716c13 (127.11.254.130:43995): tablet 00f0c2d41b7c4f07b700213fd8587a9d (table test-table [id=36d057c148e64bf89ac4c3175e96a87f]) successfully deleted
I20260812 21:03:52.892376 19708 catalog_manager.cc:4841] TS 2a3ebaf632ea40fc9aa7a5ec7254a432 (127.11.254.131:45041): tablet 00f0c2d41b7c4f07b700213fd8587a9d (table test-table [id=36d057c148e64bf89ac4c3175e96a87f]) successfully deleted
W20260812 21:03:52.894130 19708 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.254.129:36431 (error 108)
I20260812 21:03:52.894678 20546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:52.894547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.129:36431 (local address 127.0.0.1:51452)
0812 21:03:52.894600 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0812 21:03:52.894641 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.129:36431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260812 21:03:52.894733 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 20068
W20260812 21:03:52.904479 19708 catalog_manager.cc:4584] TS 94daf355993f4a3dbb64c14e00465548 (127.11.254.129:36431): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 00f0c2d41b7c4f07b700213fd8587a9d: Network error: Client connection negotiation failed: client connection to 127.11.254.129:36431: connect: Connection refused (error 111)
I20260812 21:03:52.909520 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 20322
I20260812 21:03:52.919245 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 19679
2026-08-12T21:03:52Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (3880 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-12T21:03:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T21:03:52Z Disabled control of system clock
I20260812 21:03:52.955127 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786568607354921-12282-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786568607354921-12282-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786568607354921-12282-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.254.190:44655
--webserver_interface=127.11.254.190
--webserver_port=0
--builtin_ntp_servers=127.11.254.148:45161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.254.190:44655
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260812 21:03:53.040516 20608 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:03:53.040747 20608 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:03:53.040772 20608 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:03:53.041939 20608 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 21:03:53.042004 20608 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:03:53.042016 20608 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:03:53.042028 20608 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 21:03:53.042039 20608 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 21:03:53.043408 20608 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.254.148:45161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786568607354921-12282-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786568607354921-12282-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.254.190:44655
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.254.190:44655
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.254.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20608
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786568607354921-12282-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:03:53.043645 20608 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:03:53.043907 20608 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:03:53.048820 20614 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:03:53.049109 20616 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:03:53.049289 20619 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:03:53.049469 20608 server_base.cc:1034] running on GCE node
I20260812 21:03:53.049734 20608 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:03:53.050074 20608 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:03:53.051225 20608 hybrid_clock.cc:648] HybridClock initialized: now 1786568633051207 us; error 30 us; skew 500 ppm
I20260812 21:03:53.052578 20608 webserver.cc:460] Webserver started at http://127.11.254.190:34945/ using document root <none> and password file <none>
I20260812 21:03:53.052892 20608 fs_manager.cc:359] Metadata directory not provided
I20260812 21:03:53.052963 20608 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:03:53.053110 20608 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:03:53.054380 20608 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786568607354921-12282-0/minicluster-data/master-0/data/instance:
uuid: "d0335d0f174540c982708fdf9c2abd22"
format_stamp: "Formatted at 2026-08-12 21:03:53 on dist-test-slave-vpvm"
server_key: "7579a5db6c3defac606224e878ef19cf"
server_key_iv: "682df07a8bdd7e79c4ba6907548ad67a"
server_key_version: "encryptionkey@0"
I20260812 21:03:53.054764 20608 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786568607354921-12282-0/minicluster-data/master-0/wal/instance:
uuid: "d0335d0f174540c982708fdf9c2abd22"
format_stamp: "Formatted at 2026-08-12 21:03:53 on dist-test-slave-vpvm"
server_key: "7579a5db6c3defac606224e878ef19cf"
server_key_iv: "682df07a8bdd7e79c4ba6907548ad67a"
server_key_version: "encryptionkey@0"
I20260812 21:03:53.056180 20608 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 21:03:53.057076 20628 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:03:53.057271 20608 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 21:03:53.057363 20608 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786568607354921-12282-0/minicluster-data/master-0/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786568607354921-12282-0/minicluster-data/master-0/wal
uuid: "d0335d0f174540c982708fdf9c2abd22"
format_stamp: "Formatted at 2026-08-12 21:03:53 on dist-test-slave-vpvm"
server_key: "7579a5db6c3defac606224e878ef19cf"
server_key_iv: "682df07a8bdd7e79c4ba6907548ad67a"
server_key_version: "encryptionkey@0"
I20260812 21:03:53.057431 20608 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786568607354921-12282-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786568607354921-12282-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786568607354921-12282-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:03:53.060923 20608 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:03:53.061188 20608 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:03:53.061316 20608 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:03:53.065110 20608 rpc_server.cc:307] RPC server started. Bound to: 127.11.254.190:44655
I20260812 21:03:53.065145 20729 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.254.190:44655 every 8 connection(s)
I20260812 21:03:53.065456 20608 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
I20260812 21:03:53.066244 20730 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:03:53.069308 20730 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0335d0f174540c982708fdf9c2abd22: Bootstrap starting.
I20260812 21:03:53.069739 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 20608
I20260812 21:03:53.069836 12282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786568607354921-12282-0/minicluster-data/master-0/wal/instance
I20260812 21:03:53.070013 20730 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d0335d0f174540c982708fdf9c2abd22: Neither blocks nor log segments found. Creating new log.
I20260812 21:03:53.070065 12282 external_mini_cluster.cc:1428] Setting key 5f538ff14617c5864a480ec252c533e5
I20260812 21:03:53.070309 20730 log.cc:824] T 00000000000000000000000000000000 P d0335d0f174540c982708fdf9c2abd22: Log is configured to *not* fsync() on all Append() calls
I20260812 21:03:53.071089 20730 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0335d0f174540c982708fdf9c2abd22: No bootstrap required, opened a new log
I20260812 21:03:53.072858 20735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:53.070618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42414 (local address 127.11.254.190:44655)
0812 21:03:53.070839 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0812 21:03:53.070850 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 21:03:53.070888 (+ 38us) server_negotiation.cc:407] Connection header received
0812 21:03:53.071393 (+ 505us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:03:53.071400 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:03:53.071441 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:03:53.071564 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:03:53.072252 (+ 688us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:03:53.072256 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:03:53.072256 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 21:03:53.072263 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:03:53.072319 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 21:03:53.072334 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:03:53.072423 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0812 21:03:53.072547 (+ 124us) server_negotiation.cc:299] Negotiation successful
0812 21:03:53.072695 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":86,"threads_started":1}
I20260812 21:03:53.073401 20730 raft_consensus.cc:348] T 00000000000000000000000000000000 P d0335d0f174540c982708fdf9c2abd22 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0335d0f174540c982708fdf9c2abd22" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 44655 } }
I20260812 21:03:53.073577 20730 raft_consensus.cc:374] T 00000000000000000000000000000000 P d0335d0f174540c982708fdf9c2abd22 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:03:53.073619 20730 raft_consensus.cc:729] T 00000000000000000000000000000000 P d0335d0f174540c982708fdf9c2abd22 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0335d0f174540c982708fdf9c2abd22, State: Initialized, Role: FOLLOWER
I20260812 21:03:53.073734 20730 consensus_queue.cc:260] T 00000000000000000000000000000000 P d0335d0f174540c982708fdf9c2abd22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0335d0f174540c982708fdf9c2abd22" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 44655 } }
I20260812 21:03:53.073817 20730 raft_consensus.cc:388] T 00000000000000000000000000000000 P d0335d0f174540c982708fdf9c2abd22 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 21:03:53.073849 20730 raft_consensus.cc:482] T 00000000000000000000000000000000 P d0335d0f174540c982708fdf9c2abd22 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 21:03:53.073895 20730 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d0335d0f174540c982708fdf9c2abd22 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:03:53.074777 20730 raft_consensus.cc:504] T 00000000000000000000000000000000 P d0335d0f174540c982708fdf9c2abd22 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0335d0f174540c982708fdf9c2abd22" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 44655 } }
I20260812 21:03:53.074880 20730 leader_election.cc:302] T 00000000000000000000000000000000 P d0335d0f174540c982708fdf9c2abd22 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d0335d0f174540c982708fdf9c2abd22; no voters:
I20260812 21:03:53.075050 20730 leader_election.cc:288] T 00000000000000000000000000000000 P d0335d0f174540c982708fdf9c2abd22 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 21:03:53.075098 20736 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d0335d0f174540c982708fdf9c2abd22 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:03:53.075235 20736 raft_consensus.cc:686] T 00000000000000000000000000000000 P d0335d0f174540c982708fdf9c2abd22 [term 1 LEADER]: Becoming Leader. State: Replica: d0335d0f174540c982708fdf9c2abd22, State: Running, Role: LEADER
I20260812 21:03:53.075313 20730 sys_catalog.cc:564] T 00000000000000000000000000000000 P d0335d0f174540c982708fdf9c2abd22 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 21:03:53.075345 20736 consensus_queue.cc:237] T 00000000000000000000000000000000 P d0335d0f174540c982708fdf9c2abd22 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0335d0f174540c982708fdf9c2abd22" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 44655 } }
I20260812 21:03:53.075706 20738 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0335d0f174540c982708fdf9c2abd22 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d0335d0f174540c982708fdf9c2abd22" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0335d0f174540c982708fdf9c2abd22" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 44655 } } }
I20260812 21:03:53.075826 20738 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0335d0f174540c982708fdf9c2abd22 [sys.catalog]: This master's current role is: LEADER
I20260812 21:03:53.075687 20739 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0335d0f174540c982708fdf9c2abd22 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d0335d0f174540c982708fdf9c2abd22. Latest consensus state: current_term: 1 leader_uuid: "d0335d0f174540c982708fdf9c2abd22" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0335d0f174540c982708fdf9c2abd22" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 44655 } } }
I20260812 21:03:53.075984 20739 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0335d0f174540c982708fdf9c2abd22 [sys.catalog]: This master's current role is: LEADER
I20260812 21:03:53.076236 20748 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 21:03:53.076697 20748 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 21:03:53.078194 20748 catalog_manager.cc:1347] Generated new cluster ID: 3176b4f28da0428684ca50c385a0a7de
I20260812 21:03:53.078243 20748 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 21:03:53.100785 20748 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 21:03:53.100993 20748 catalog_manager.cc:1504] Loading token signing keys...
I20260812 21:03:53.126539 20748 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d0335d0f174540c982708fdf9c2abd22: Generated new TSK 0
I20260812 21:03:53.126801 20748 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 21:03:53.136240 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786568607354921-12282-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786568607354921-12282-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786568607354921-12282-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786568607354921-12282-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.254.129:0
--local_ip_for_outbound_sockets=127.11.254.129
--webserver_interface=127.11.254.129
--webserver_port=0
--tserver_master_addrs=127.11.254.190:44655
--builtin_ntp_servers=127.11.254.148:45161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260812 21:03:53.221539 20771 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:03:53.221743 20771 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:03:53.221766 20771 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:03:53.222930 20771 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:03:53.222970 20771 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:03:53.223001 20771 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.254.129
I20260812 21:03:53.224334 20771 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.254.148:45161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786568607354921-12282-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786568607354921-12282-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.254.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786568607354921-12282-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.254.129
--webserver_port=0
--tserver_master_addrs=127.11.254.190:44655
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20771
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.254.129
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786568607354921-12282-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:03:53.224538 20771 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:03:53.224797 20771 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:03:53.229977 20783 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:03:53.230067 20771 server_base.cc:1034] running on GCE node
W20260812 21:03:53.230036 20779 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:03:53.230005 20780 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:03:53.230358 20771 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:03:53.230634 20771 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:03:53.231796 20771 hybrid_clock.cc:648] HybridClock initialized: now 1786568633231782 us; error 30 us; skew 500 ppm
I20260812 21:03:53.233109 20771 webserver.cc:460] Webserver started at http://127.11.254.129:45639/ using document root <none> and password file <none>
I20260812 21:03:53.233353 20771 fs_manager.cc:359] Metadata directory not provided
I20260812 21:03:53.233405 20771 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:03:53.233517 20771 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:03:53.234644 20771 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786568607354921-12282-0/minicluster-data/ts-0/data/instance:
uuid: "d756bc97f1a94cec9d79612302f2266a"
format_stamp: "Formatted at 2026-08-12 21:03:53 on dist-test-slave-vpvm"
server_key: "d229ede8aa43d6c2c327d2fd2bc13201"
server_key_iv: "4a932096d401898eda7ec5af2c03adc6"
server_key_version: "encryptionkey@0"
I20260812 21:03:53.235073 20771 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786568607354921-12282-0/minicluster-data/ts-0/wal/instance:
uuid: "d756bc97f1a94cec9d79612302f2266a"
format_stamp: "Formatted at 2026-08-12 21:03:53 on dist-test-slave-vpvm"
server_key: "d229ede8aa43d6c2c327d2fd2bc13201"
server_key_iv: "4a932096d401898eda7ec5af2c03adc6"
server_key_version: "encryptionkey@0"
I20260812 21:03:53.236370 20771 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 21:03:53.237169 20795 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:03:53.237340 20771 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 21:03:53.237439 20771 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786568607354921-12282-0/minicluster-data/ts-0/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786568607354921-12282-0/minicluster-data/ts-0/wal
uuid: "d756bc97f1a94cec9d79612302f2266a"
format_stamp: "Formatted at 2026-08-12 21:03:53 on dist-test-slave-vpvm"
server_key: "d229ede8aa43d6c2c327d2fd2bc13201"
server_key_iv: "4a932096d401898eda7ec5af2c03adc6"
server_key_version: "encryptionkey@0"
I20260812 21:03:53.237504 20771 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786568607354921-12282-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786568607354921-12282-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786568607354921-12282-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:03:53.240942 20771 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:03:53.241209 20771 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:03:53.241335 20771 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:03:53.241531 20771 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:03:53.241824 20771 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:03:53.241858 20771 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:03:53.241886 20771 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:03:53.241918 20771 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:03:53.247594 20771 rpc_server.cc:307] RPC server started. Bound to: 127.11.254.129:37147
I20260812 21:03:53.247669 20988 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.254.129:37147 every 8 connection(s)
I20260812 21:03:53.247987 20771 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786568607354921-12282-0/minicluster-data/ts-0/data/info.pb
I20260812 21:03:53.251116 20735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:53.248782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.129:52301 (local address 127.11.254.190:44655)
0812 21:03:53.248848 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0812 21:03:53.248850 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:03:53.249075 (+ 225us) server_negotiation.cc:407] Connection header received
0812 21:03:53.249581 (+ 506us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:03:53.249582 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:03:53.249590 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:03:53.249643 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:03:53.250854 (+ 1211us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:03:53.250856 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:03:53.250856 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 21:03:53.250861 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:03:53.250878 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 21:03:53.250883 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:03:53.250934 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0812 21:03:53.251005 (+ 71us) server_negotiation.cc:299] Negotiation successful
0812 21:03:53.251020 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260812 21:03:53.251291 20994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:53.248754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:44655 (local address 127.11.254.129:52301)
0812 21:03:53.248956 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0812 21:03:53.248990 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0812 21:03:53.249469 (+ 479us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:03:53.249715 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:03:53.249720 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:03:53.249729 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:03:53.250701 (+ 972us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 21:03:53.250702 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:03:53.250731 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 21:03:53.250761 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:03:53.250937 (+ 176us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:03:53.250942 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:03:53.250945 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 21:03:53.251012 (+ 67us) client_negotiation.cc:240] Negotiation successful
0812 21:03:53.251078 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":97,"threads_started":1}
I20260812 21:03:53.251776 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 20771
I20260812 21:03:53.251814 20989 heartbeater.cc:347] Connected to a master server at 127.11.254.190:44655
I20260812 21:03:53.251878 12282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786568607354921-12282-0/minicluster-data/ts-0/wal/instance
I20260812 21:03:53.251901 20989 heartbeater.cc:464] Registering TS with master...
I20260812 21:03:53.252072 20989 heartbeater.cc:511] Master 127.11.254.190:44655 requested a full tablet report, sending...
I20260812 21:03:53.252077 12282 external_mini_cluster.cc:1428] Setting key f803c7c28069fce8e90df8d701eb182b
I20260812 21:03:53.252585 20654 ts_manager.cc:194] Registered new tserver with Master: d756bc97f1a94cec9d79612302f2266a (127.11.254.129:37147)
I20260812 21:03:53.253090 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786568607354921-12282-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786568607354921-12282-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786568607354921-12282-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786568607354921-12282-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.254.130:0
--local_ip_for_outbound_sockets=127.11.254.130
--webserver_interface=127.11.254.130
--webserver_port=0
--tserver_master_addrs=127.11.254.190:44655
--builtin_ntp_servers=127.11.254.148:45161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260812 21:03:53.336124 20997 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:03:53.336345 20997 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:03:53.336367 20997 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:03:53.337560 20997 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:03:53.337606 20997 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:03:53.337634 20997 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.254.130
I20260812 21:03:53.339439 20997 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.254.148:45161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786568607354921-12282-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786568607354921-12282-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.254.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786568607354921-12282-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.254.130
--webserver_port=0
--tserver_master_addrs=127.11.254.190:44655
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20997
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.254.130
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786568607354921-12282-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:03:53.339697 20997 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:03:53.339989 20997 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:03:53.345077 21017 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:03:53.345049 21011 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:03:53.345085 21010 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:03:53.345324 20997 server_base.cc:1034] running on GCE node
I20260812 21:03:53.345528 20997 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:03:53.345742 20997 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:03:53.346887 20997 hybrid_clock.cc:648] HybridClock initialized: now 1786568633346874 us; error 27 us; skew 500 ppm
I20260812 21:03:53.348161 20997 webserver.cc:460] Webserver started at http://127.11.254.130:46881/ using document root <none> and password file <none>
I20260812 21:03:53.348416 20997 fs_manager.cc:359] Metadata directory not provided
I20260812 21:03:53.348462 20997 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:03:53.348640 20997 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:03:53.349726 20997 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786568607354921-12282-0/minicluster-data/ts-1/data/instance:
uuid: "c3ded1c502a64427aae188ab512ba6db"
format_stamp: "Formatted at 2026-08-12 21:03:53 on dist-test-slave-vpvm"
server_key: "b395d15eab5e49a0d83a93046d324109"
server_key_iv: "087afe1d62ae9a453387ebb00cbfe433"
server_key_version: "encryptionkey@0"
I20260812 21:03:53.350101 20997 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786568607354921-12282-0/minicluster-data/ts-1/wal/instance:
uuid: "c3ded1c502a64427aae188ab512ba6db"
format_stamp: "Formatted at 2026-08-12 21:03:53 on dist-test-slave-vpvm"
server_key: "b395d15eab5e49a0d83a93046d324109"
server_key_iv: "087afe1d62ae9a453387ebb00cbfe433"
server_key_version: "encryptionkey@0"
I20260812 21:03:53.351373 20997 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 21:03:53.352092 21023 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:03:53.352260 20997 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 21:03:53.352346 20997 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786568607354921-12282-0/minicluster-data/ts-1/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786568607354921-12282-0/minicluster-data/ts-1/wal
uuid: "c3ded1c502a64427aae188ab512ba6db"
format_stamp: "Formatted at 2026-08-12 21:03:53 on dist-test-slave-vpvm"
server_key: "b395d15eab5e49a0d83a93046d324109"
server_key_iv: "087afe1d62ae9a453387ebb00cbfe433"
server_key_version: "encryptionkey@0"
I20260812 21:03:53.352409 20997 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786568607354921-12282-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786568607354921-12282-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786568607354921-12282-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:03:53.355831 20997 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:03:53.356076 20997 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:03:53.356196 20997 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:03:53.356376 20997 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:03:53.356678 20997 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:03:53.356712 20997 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:03:53.356739 20997 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:03:53.356763 20997 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:03:53.362409 20997 rpc_server.cc:307] RPC server started. Bound to: 127.11.254.130:36923
I20260812 21:03:53.362449 21211 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.254.130:36923 every 8 connection(s)
I20260812 21:03:53.362792 20997 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786568607354921-12282-0/minicluster-data/ts-1/data/info.pb
I20260812 21:03:53.365605 20735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:53.363480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.130:54341 (local address 127.11.254.190:44655)
0812 21:03:53.363562 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0812 21:03:53.363565 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 21:03:53.363779 (+ 214us) server_negotiation.cc:407] Connection header received
0812 21:03:53.364275 (+ 496us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:03:53.364276 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:03:53.364283 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:03:53.364341 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:03:53.365417 (+ 1076us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:03:53.365417 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:03:53.365418 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 21:03:53.365421 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:03:53.365433 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 21:03:53.365436 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:03:53.365474 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0812 21:03:53.365533 (+ 59us) server_negotiation.cc:299] Negotiation successful
0812 21:03:53.365547 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260812 21:03:53.365770 21217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:53.363468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:44655 (local address 127.11.254.130:54341)
0812 21:03:53.363663 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0812 21:03:53.363698 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0812 21:03:53.364167 (+ 469us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:03:53.364395 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:03:53.364398 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:03:53.364406 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:03:53.365283 (+ 877us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 21:03:53.365284 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:03:53.365313 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 21:03:53.365344 (+ 31us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:03:53.365479 (+ 135us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:03:53.365481 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:03:53.365484 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 21:03:53.365532 (+ 48us) client_negotiation.cc:240] Negotiation successful
0812 21:03:53.365584 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":95,"threads_started":1}
I20260812 21:03:53.366251 21214 heartbeater.cc:347] Connected to a master server at 127.11.254.190:44655
I20260812 21:03:53.366334 21214 heartbeater.cc:464] Registering TS with master...
I20260812 21:03:53.366454 21214 heartbeater.cc:511] Master 127.11.254.190:44655 requested a full tablet report, sending...
I20260812 21:03:53.366776 20654 ts_manager.cc:194] Registered new tserver with Master: c3ded1c502a64427aae188ab512ba6db (127.11.254.130:36923)
I20260812 21:03:53.368714 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 20997
I20260812 21:03:53.368809 12282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786568607354921-12282-0/minicluster-data/ts-1/wal/instance
I20260812 21:03:53.368984 12282 external_mini_cluster.cc:1428] Setting key 99bffb748174638af210b92e47186b23
I20260812 21:03:53.370045 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786568607354921-12282-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786568607354921-12282-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786568607354921-12282-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786568607354921-12282-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.254.131:0
--local_ip_for_outbound_sockets=127.11.254.131
--webserver_interface=127.11.254.131
--webserver_port=0
--tserver_master_addrs=127.11.254.190:44655
--builtin_ntp_servers=127.11.254.148:45161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260812 21:03:53.452338 21218 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:03:53.452556 21218 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:03:53.452579 21218 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:03:53.453758 21218 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:03:53.453804 21218 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:03:53.453830 21218 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.254.131
I20260812 21:03:53.455264 21218 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.254.148:45161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786568607354921-12282-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786568607354921-12282-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.254.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786568607354921-12282-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.254.131
--webserver_port=0
--tserver_master_addrs=127.11.254.190:44655
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21218
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.254.131
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786568607354921-12282-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:03:53.455511 21218 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:03:53.455804 21218 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:03:53.460825 21228 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:03:53.460825 21230 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:03:53.461078 21218 server_base.cc:1034] running on GCE node
W20260812 21:03:53.461165 21225 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:03:53.461386 21218 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:03:53.461619 21218 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:03:53.462850 21218 hybrid_clock.cc:648] HybridClock initialized: now 1786568633462825 us; error 35 us; skew 500 ppm
I20260812 21:03:53.464219 21218 webserver.cc:460] Webserver started at http://127.11.254.131:39355/ using document root <none> and password file <none>
I20260812 21:03:53.464450 21218 fs_manager.cc:359] Metadata directory not provided
I20260812 21:03:53.464501 21218 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:03:53.464691 21218 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:03:53.465770 21218 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786568607354921-12282-0/minicluster-data/ts-2/data/instance:
uuid: "2b8213e6b65a43a7880293fbf5ce7f74"
format_stamp: "Formatted at 2026-08-12 21:03:53 on dist-test-slave-vpvm"
server_key: "95663f65f91ac47ea94f6085ff9fcc1d"
server_key_iv: "bf0900d4153c00f0d00a94caac534a76"
server_key_version: "encryptionkey@0"
I20260812 21:03:53.466144 21218 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786568607354921-12282-0/minicluster-data/ts-2/wal/instance:
uuid: "2b8213e6b65a43a7880293fbf5ce7f74"
format_stamp: "Formatted at 2026-08-12 21:03:53 on dist-test-slave-vpvm"
server_key: "95663f65f91ac47ea94f6085ff9fcc1d"
server_key_iv: "bf0900d4153c00f0d00a94caac534a76"
server_key_version: "encryptionkey@0"
I20260812 21:03:53.467357 21218 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 21:03:53.468137 21240 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:03:53.468294 21218 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 21:03:53.468384 21218 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786568607354921-12282-0/minicluster-data/ts-2/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786568607354921-12282-0/minicluster-data/ts-2/wal
uuid: "2b8213e6b65a43a7880293fbf5ce7f74"
format_stamp: "Formatted at 2026-08-12 21:03:53 on dist-test-slave-vpvm"
server_key: "95663f65f91ac47ea94f6085ff9fcc1d"
server_key_iv: "bf0900d4153c00f0d00a94caac534a76"
server_key_version: "encryptionkey@0"
I20260812 21:03:53.468456 21218 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786568607354921-12282-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786568607354921-12282-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786568607354921-12282-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:03:53.471901 21218 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:03:53.472151 21218 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:03:53.472271 21218 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:03:53.472457 21218 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:03:53.472761 21218 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:03:53.472801 21218 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:03:53.472828 21218 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:03:53.472851 21218 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:03:53.478463 21218 rpc_server.cc:307] RPC server started. Bound to: 127.11.254.131:43863
I20260812 21:03:53.478502 21445 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.254.131:43863 every 8 connection(s)
I20260812 21:03:53.478858 21218 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786568607354921-12282-0/minicluster-data/ts-2/data/info.pb
I20260812 21:03:53.481817 20735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:53.479631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.131:56611 (local address 127.11.254.190:44655)
0812 21:03:53.479721 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0812 21:03:53.479724 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 21:03:53.479906 (+ 182us) server_negotiation.cc:407] Connection header received
0812 21:03:53.480394 (+ 488us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:03:53.480395 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:03:53.480402 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:03:53.480448 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:03:53.481632 (+ 1184us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:03:53.481633 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:03:53.481634 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 21:03:53.481637 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:03:53.481653 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 21:03:53.481657 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:03:53.481701 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0812 21:03:53.481749 (+ 48us) server_negotiation.cc:299] Negotiation successful
0812 21:03:53.481761 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260812 21:03:53.482033 21449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:53.479566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:44655 (local address 127.11.254.131:56611)
0812 21:03:53.479752 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0812 21:03:53.479789 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0812 21:03:53.480285 (+ 496us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:03:53.480518 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:03:53.480522 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:03:53.480530 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:03:53.481478 (+ 948us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 21:03:53.481479 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:03:53.481510 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 21:03:53.481543 (+ 33us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:03:53.481702 (+ 159us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:03:53.481704 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:03:53.481707 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 21:03:53.481757 (+ 50us) client_negotiation.cc:240] Negotiation successful
0812 21:03:53.481809 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":72,"threads_started":1}
I20260812 21:03:53.482503 21446 heartbeater.cc:347] Connected to a master server at 127.11.254.190:44655
I20260812 21:03:53.482604 21446 heartbeater.cc:464] Registering TS with master...
I20260812 21:03:53.482791 21446 heartbeater.cc:511] Master 127.11.254.190:44655 requested a full tablet report, sending...
I20260812 21:03:53.483207 20654 ts_manager.cc:194] Registered new tserver with Master: 2b8213e6b65a43a7880293fbf5ce7f74 (127.11.254.131:43863)
I20260812 21:03:53.485365 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 21218
I20260812 21:03:53.485445 12282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786568607354921-12282-0/minicluster-data/ts-2/wal/instance
I20260812 21:03:53.485586 12282 external_mini_cluster.cc:1428] Setting key bf4c154fd330ee5483654aafd5b5e637
I20260812 21:03:53.486433 12282 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 21:03:53.488827 20735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:53.487748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42428 (local address 127.11.254.190:44655)
0812 21:03:53.487798 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0812 21:03:53.487800 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:03:53.487906 (+ 106us) server_negotiation.cc:407] Connection header received
0812 21:03:53.487979 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:03:53.487980 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:03:53.487985 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:03:53.488039 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:03:53.488619 (+ 580us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:03:53.488620 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:03:53.488620 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 21:03:53.488625 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:03:53.488640 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 21:03:53.488644 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:03:53.488692 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0812 21:03:53.488739 (+ 47us) server_negotiation.cc:299] Negotiation successful
0812 21:03:53.488753 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260812 21:03:53.490374 20654 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:42428:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 21:03:53.491875 20654 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 21:03:53.495626 21460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:53.494064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.129:37147 (local address 127.0.0.1:46000)
0812 21:03:53.494304 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0812 21:03:53.494441 (+ 137us) client_negotiation.cc:174] Beginning negotiation
0812 21:03:53.494500 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:03:53.494726 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:03:53.494733 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:03:53.494739 (+ 6us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:03:53.495210 (+ 471us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 21:03:53.495210 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:03:53.495229 (+ 19us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 21:03:53.495254 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:03:53.495475 (+ 221us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:03:53.495483 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:03:53.495486 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 21:03:53.495531 (+ 45us) client_negotiation.cc:240] Negotiation successful
0812 21:03:53.495545 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":55,"threads_started":1}
I20260812 21:03:53.495946 21462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:53.494334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46000 (local address 127.11.254.129:37147)
0812 21:03:53.494602 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0812 21:03:53.494606 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 21:03:53.494615 (+ 9us) server_negotiation.cc:407] Connection header received
0812 21:03:53.494640 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:03:53.494641 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:03:53.494649 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:03:53.494715 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:03:53.495317 (+ 602us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:03:53.495327 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:03:53.495327 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 21:03:53.495331 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:03:53.495368 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 21:03:53.495393 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:03:53.495822 (+ 429us) server_negotiation.cc:1035] Waiting for connection context
0812 21:03:53.495848 (+ 26us) server_negotiation.cc:299] Negotiation successful
0812 21:03:53.495864 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":225,"thread_start_us":117,"threads_started":1}
I20260812 21:03:53.495967 21456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:53.494140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.131:43863 (local address 127.0.0.1:33072)
0812 21:03:53.494469 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0812 21:03:53.494479 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 21:03:53.494521 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:03:53.495128 (+ 607us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:03:53.495129 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:03:53.495132 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:03:53.495604 (+ 472us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 21:03:53.495604 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:03:53.495610 (+ 6us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 21:03:53.495621 (+ 11us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:03:53.495854 (+ 233us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:03:53.495854 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:03:53.495855 (+ 1us) client_negotiation.cc:769] Sending connection context
0812 21:03:53.495904 (+ 49us) client_negotiation.cc:240] Negotiation successful
0812 21:03:53.495915 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":316,"thread_start_us":100,"threads_started":1}
I20260812 21:03:53.496191 21463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:53.494479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57990 (local address 127.11.254.130:36923)
0812 21:03:53.494678 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0812 21:03:53.494682 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 21:03:53.495151 (+ 469us) server_negotiation.cc:407] Connection header received
0812 21:03:53.495181 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:03:53.495183 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:03:53.495193 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:03:53.495265 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:03:53.495871 (+ 606us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:03:53.495878 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:03:53.495879 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 21:03:53.495883 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:03:53.495917 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 21:03:53.495943 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:03:53.495992 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0812 21:03:53.496086 (+ 94us) server_negotiation.cc:299] Negotiation successful
0812 21:03:53.496111 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":100,"threads_started":1}
I20260812 21:03:53.496238 21459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:53.494144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33072 (local address 127.11.254.131:43863)
0812 21:03:53.494321 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0812 21:03:53.494326 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 21:03:53.494664 (+ 338us) server_negotiation.cc:407] Connection header received
0812 21:03:53.494751 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:03:53.494756 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:03:53.494766 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:03:53.494835 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:03:53.495704 (+ 869us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:03:53.495711 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:03:53.495711 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 21:03:53.495716 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:03:53.495752 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 21:03:53.495780 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:03:53.495830 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0812 21:03:53.495921 (+ 91us) server_negotiation.cc:299] Negotiation successful
0812 21:03:53.495939 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":91,"threads_started":1}
I20260812 21:03:53.496490 21461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:53.494348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.130:36923 (local address 127.0.0.1:57990)
0812 21:03:53.495042 (+ 694us) negotiation.cc:107] Waiting for socket to connect
0812 21:03:53.495050 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0812 21:03:53.495094 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:03:53.495359 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:03:53.495361 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:03:53.495363 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:03:53.495774 (+ 411us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 21:03:53.495774 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:03:53.495781 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 21:03:53.495794 (+ 13us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:03:53.496015 (+ 221us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:03:53.496015 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:03:53.496016 (+ 1us) client_negotiation.cc:769] Sending connection context
0812 21:03:53.496064 (+ 48us) client_negotiation.cc:240] Negotiation successful
0812 21:03:53.496077 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":681,"thread_start_us":67,"threads_started":1}
I20260812 21:03:53.496742 20882 tablet_service.cc:1467] Processing CreateTablet for tablet c0b450c8fe3f45c190bacf1c14fa0d4a (DEFAULT_TABLE table=test-table [id=e8f75efb3d9344d68e8b47d1c796d2d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:03:53.496860 21315 tablet_service.cc:1467] Processing CreateTablet for tablet c0b450c8fe3f45c190bacf1c14fa0d4a (DEFAULT_TABLE table=test-table [id=e8f75efb3d9344d68e8b47d1c796d2d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:03:53.497143 20882 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c0b450c8fe3f45c190bacf1c14fa0d4a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:03:53.497215 21315 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c0b450c8fe3f45c190bacf1c14fa0d4a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:03:53.497133 21098 tablet_service.cc:1467] Processing CreateTablet for tablet c0b450c8fe3f45c190bacf1c14fa0d4a (DEFAULT_TABLE table=test-table [id=e8f75efb3d9344d68e8b47d1c796d2d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:03:53.497419 21098 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c0b450c8fe3f45c190bacf1c14fa0d4a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:03:53.499819 21465 tablet_bootstrap.cc:492] T c0b450c8fe3f45c190bacf1c14fa0d4a P 2b8213e6b65a43a7880293fbf5ce7f74: Bootstrap starting.
I20260812 21:03:53.500293 21466 tablet_bootstrap.cc:492] T c0b450c8fe3f45c190bacf1c14fa0d4a P c3ded1c502a64427aae188ab512ba6db: Bootstrap starting.
I20260812 21:03:53.500550 21465 tablet_bootstrap.cc:654] T c0b450c8fe3f45c190bacf1c14fa0d4a P 2b8213e6b65a43a7880293fbf5ce7f74: Neither blocks nor log segments found. Creating new log.
I20260812 21:03:53.500588 21470 tablet_bootstrap.cc:492] T c0b450c8fe3f45c190bacf1c14fa0d4a P d756bc97f1a94cec9d79612302f2266a: Bootstrap starting.
I20260812 21:03:53.500869 21465 log.cc:824] T c0b450c8fe3f45c190bacf1c14fa0d4a P 2b8213e6b65a43a7880293fbf5ce7f74: Log is configured to *not* fsync() on all Append() calls
I20260812 21:03:53.500897 21466 tablet_bootstrap.cc:654] T c0b450c8fe3f45c190bacf1c14fa0d4a P c3ded1c502a64427aae188ab512ba6db: Neither blocks nor log segments found. Creating new log.
I20260812 21:03:53.501134 21470 tablet_bootstrap.cc:654] T c0b450c8fe3f45c190bacf1c14fa0d4a P d756bc97f1a94cec9d79612302f2266a: Neither blocks nor log segments found. Creating new log.
I20260812 21:03:53.501181 21466 log.cc:824] T c0b450c8fe3f45c190bacf1c14fa0d4a P c3ded1c502a64427aae188ab512ba6db: Log is configured to *not* fsync() on all Append() calls
I20260812 21:03:53.501525 21470 log.cc:824] T c0b450c8fe3f45c190bacf1c14fa0d4a P d756bc97f1a94cec9d79612302f2266a: Log is configured to *not* fsync() on all Append() calls
I20260812 21:03:53.501650 21465 tablet_bootstrap.cc:492] T c0b450c8fe3f45c190bacf1c14fa0d4a P 2b8213e6b65a43a7880293fbf5ce7f74: No bootstrap required, opened a new log
I20260812 21:03:53.501722 21465 ts_tablet_manager.cc:1397] T c0b450c8fe3f45c190bacf1c14fa0d4a P 2b8213e6b65a43a7880293fbf5ce7f74: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 21:03:53.501893 21466 tablet_bootstrap.cc:492] T c0b450c8fe3f45c190bacf1c14fa0d4a P c3ded1c502a64427aae188ab512ba6db: No bootstrap required, opened a new log
I20260812 21:03:53.501981 21466 ts_tablet_manager.cc:1397] T c0b450c8fe3f45c190bacf1c14fa0d4a P c3ded1c502a64427aae188ab512ba6db: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 21:03:53.504379 21466 raft_consensus.cc:348] T c0b450c8fe3f45c190bacf1c14fa0d4a P c3ded1c502a64427aae188ab512ba6db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b8213e6b65a43a7880293fbf5ce7f74" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 43863 } } peers { permanent_uuid: "d756bc97f1a94cec9d79612302f2266a" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 37147 } } peers { permanent_uuid: "c3ded1c502a64427aae188ab512ba6db" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 36923 } }
I20260812 21:03:53.504340 21465 raft_consensus.cc:348] T c0b450c8fe3f45c190bacf1c14fa0d4a P 2b8213e6b65a43a7880293fbf5ce7f74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b8213e6b65a43a7880293fbf5ce7f74" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 43863 } } peers { permanent_uuid: "d756bc97f1a94cec9d79612302f2266a" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 37147 } } peers { permanent_uuid: "c3ded1c502a64427aae188ab512ba6db" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 36923 } }
I20260812 21:03:53.504519 21466 raft_consensus.cc:374] T c0b450c8fe3f45c190bacf1c14fa0d4a P c3ded1c502a64427aae188ab512ba6db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:03:53.504519 21465 raft_consensus.cc:374] T c0b450c8fe3f45c190bacf1c14fa0d4a P 2b8213e6b65a43a7880293fbf5ce7f74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:03:53.504554 21466 raft_consensus.cc:729] T c0b450c8fe3f45c190bacf1c14fa0d4a P c3ded1c502a64427aae188ab512ba6db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3ded1c502a64427aae188ab512ba6db, State: Initialized, Role: FOLLOWER
I20260812 21:03:53.504554 21465 raft_consensus.cc:729] T c0b450c8fe3f45c190bacf1c14fa0d4a P 2b8213e6b65a43a7880293fbf5ce7f74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b8213e6b65a43a7880293fbf5ce7f74, State: Initialized, Role: FOLLOWER
I20260812 21:03:53.504673 21465 consensus_queue.cc:260] T c0b450c8fe3f45c190bacf1c14fa0d4a P 2b8213e6b65a43a7880293fbf5ce7f74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b8213e6b65a43a7880293fbf5ce7f74" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 43863 } } peers { permanent_uuid: "d756bc97f1a94cec9d79612302f2266a" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 37147 } } peers { permanent_uuid: "c3ded1c502a64427aae188ab512ba6db" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 36923 } }
I20260812 21:03:53.504673 21466 consensus_queue.cc:260] T c0b450c8fe3f45c190bacf1c14fa0d4a P c3ded1c502a64427aae188ab512ba6db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b8213e6b65a43a7880293fbf5ce7f74" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 43863 } } peers { permanent_uuid: "d756bc97f1a94cec9d79612302f2266a" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 37147 } } peers { permanent_uuid: "c3ded1c502a64427aae188ab512ba6db" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 36923 } }
I20260812 21:03:53.504935 21466 ts_tablet_manager.cc:1428] T c0b450c8fe3f45c190bacf1c14fa0d4a P c3ded1c502a64427aae188ab512ba6db: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 21:03:53.504935 21465 ts_tablet_manager.cc:1428] T c0b450c8fe3f45c190bacf1c14fa0d4a P 2b8213e6b65a43a7880293fbf5ce7f74: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 21:03:53.505036 21446 heartbeater.cc:503] Master 127.11.254.190:44655 was elected leader, sending a full tablet report...
I20260812 21:03:53.504963 21214 heartbeater.cc:503] Master 127.11.254.190:44655 was elected leader, sending a full tablet report...
I20260812 21:03:53.505940 21470 tablet_bootstrap.cc:492] T c0b450c8fe3f45c190bacf1c14fa0d4a P d756bc97f1a94cec9d79612302f2266a: No bootstrap required, opened a new log
I20260812 21:03:53.506047 21470 ts_tablet_manager.cc:1397] T c0b450c8fe3f45c190bacf1c14fa0d4a P d756bc97f1a94cec9d79612302f2266a: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260812 21:03:53.507871 21470 raft_consensus.cc:348] T c0b450c8fe3f45c190bacf1c14fa0d4a P d756bc97f1a94cec9d79612302f2266a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b8213e6b65a43a7880293fbf5ce7f74" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 43863 } } peers { permanent_uuid: "d756bc97f1a94cec9d79612302f2266a" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 37147 } } peers { permanent_uuid: "c3ded1c502a64427aae188ab512ba6db" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 36923 } }
I20260812 21:03:53.507989 21470 raft_consensus.cc:374] T c0b450c8fe3f45c190bacf1c14fa0d4a P d756bc97f1a94cec9d79612302f2266a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:03:53.508008 21470 raft_consensus.cc:729] T c0b450c8fe3f45c190bacf1c14fa0d4a P d756bc97f1a94cec9d79612302f2266a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d756bc97f1a94cec9d79612302f2266a, State: Initialized, Role: FOLLOWER
I20260812 21:03:53.508096 21470 consensus_queue.cc:260] T c0b450c8fe3f45c190bacf1c14fa0d4a P d756bc97f1a94cec9d79612302f2266a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b8213e6b65a43a7880293fbf5ce7f74" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 43863 } } peers { permanent_uuid: "d756bc97f1a94cec9d79612302f2266a" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 37147 } } peers { permanent_uuid: "c3ded1c502a64427aae188ab512ba6db" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 36923 } }
I20260812 21:03:53.508291 21470 ts_tablet_manager.cc:1428] T c0b450c8fe3f45c190bacf1c14fa0d4a P d756bc97f1a94cec9d79612302f2266a: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260812 21:03:53.508342 20989 heartbeater.cc:503] Master 127.11.254.190:44655 was elected leader, sending a full tablet report...
W20260812 21:03:53.613334 21216 tablet.cc:2369] T c0b450c8fe3f45c190bacf1c14fa0d4a P c3ded1c502a64427aae188ab512ba6db: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 21:03:53.636328 21477 raft_consensus.cc:482] T c0b450c8fe3f45c190bacf1c14fa0d4a P 2b8213e6b65a43a7880293fbf5ce7f74 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 21:03:53.636466 21477 raft_consensus.cc:504] T c0b450c8fe3f45c190bacf1c14fa0d4a P 2b8213e6b65a43a7880293fbf5ce7f74 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b8213e6b65a43a7880293fbf5ce7f74" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 43863 } } peers { permanent_uuid: "d756bc97f1a94cec9d79612302f2266a" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 37147 } } peers { permanent_uuid: "c3ded1c502a64427aae188ab512ba6db" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 36923 } }
I20260812 21:03:53.636852 21477 leader_election.cc:288] T c0b450c8fe3f45c190bacf1c14fa0d4a P 2b8213e6b65a43a7880293fbf5ce7f74 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d756bc97f1a94cec9d79612302f2266a (127.11.254.129:37147), c3ded1c502a64427aae188ab512ba6db (127.11.254.130:36923)
I20260812 21:03:53.638134 21449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:53.637072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.130:36923 (local address 127.11.254.131:33481)
0812 21:03:53.637134 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0812 21:03:53.637150 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 21:03:53.637210 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:03:53.637350 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:03:53.637351 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:03:53.637354 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:03:53.637794 (+ 440us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 21:03:53.637795 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:03:53.637804 (+ 9us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 21:03:53.637819 (+ 15us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:03:53.638030 (+ 211us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:03:53.638031 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:03:53.638032 (+ 1us) client_negotiation.cc:769] Sending connection context
0812 21:03:53.638071 (+ 39us) client_negotiation.cc:240] Negotiation successful
0812 21:03:53.638083 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260812 21:03:53.638288 21463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:53.637152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.131:33481 (local address 127.11.254.130:36923)
0812 21:03:53.637204 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0812 21:03:53.637206 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:03:53.637221 (+ 15us) server_negotiation.cc:407] Connection header received
0812 21:03:53.637285 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:03:53.637286 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:03:53.637292 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:03:53.637341 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:03:53.637900 (+ 559us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:03:53.637902 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:03:53.637902 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 21:03:53.637906 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:03:53.637925 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 21:03:53.637929 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:03:53.638032 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0812 21:03:53.638203 (+ 171us) server_negotiation.cc:299] Negotiation successful
0812 21:03:53.638218 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260812 21:03:53.638314 21462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:53.637239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.131:38825 (local address 127.11.254.129:37147)
0812 21:03:53.637284 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0812 21:03:53.637285 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 21:03:53.637468 (+ 183us) server_negotiation.cc:407] Connection header received
0812 21:03:53.637559 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:03:53.637560 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:03:53.637565 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:03:53.637613 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:03:53.638101 (+ 488us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:03:53.638102 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:03:53.638103 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 21:03:53.638107 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:03:53.638123 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 21:03:53.638127 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:03:53.638177 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0812 21:03:53.638247 (+ 70us) server_negotiation.cc:299] Negotiation successful
0812 21:03:53.638260 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 21:03:53.638504 21481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:53.637075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.129:37147 (local address 127.11.254.131:38825)
0812 21:03:53.637396 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0812 21:03:53.637405 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 21:03:53.637460 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:03:53.637629 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:03:53.637629 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:03:53.637645 (+ 16us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:03:53.637991 (+ 346us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 21:03:53.637991 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:03:53.637998 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 21:03:53.638009 (+ 11us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:03:53.638179 (+ 170us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:03:53.638180 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:03:53.638181 (+ 1us) client_negotiation.cc:769] Sending connection context
0812 21:03:53.638414 (+ 233us) client_negotiation.cc:240] Negotiation successful
0812 21:03:53.638428 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307,"mutex_wait_us":156,"thread_start_us":105,"threads_started":1}
I20260812 21:03:53.638612 21137 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c0b450c8fe3f45c190bacf1c14fa0d4a" candidate_uuid: "2b8213e6b65a43a7880293fbf5ce7f74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3ded1c502a64427aae188ab512ba6db" is_pre_election: true
I20260812 21:03:53.638777 21137 raft_consensus.cc:2454] T c0b450c8fe3f45c190bacf1c14fa0d4a P c3ded1c502a64427aae188ab512ba6db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b8213e6b65a43a7880293fbf5ce7f74 in term 0.
I20260812 21:03:53.638821 20907 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c0b450c8fe3f45c190bacf1c14fa0d4a" candidate_uuid: "2b8213e6b65a43a7880293fbf5ce7f74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d756bc97f1a94cec9d79612302f2266a" is_pre_election: true
I20260812 21:03:53.638980 20907 raft_consensus.cc:2454] T c0b450c8fe3f45c190bacf1c14fa0d4a P d756bc97f1a94cec9d79612302f2266a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b8213e6b65a43a7880293fbf5ce7f74 in term 0.
I20260812 21:03:53.639008 21241 leader_election.cc:302] T c0b450c8fe3f45c190bacf1c14fa0d4a P 2b8213e6b65a43a7880293fbf5ce7f74 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b8213e6b65a43a7880293fbf5ce7f74, c3ded1c502a64427aae188ab512ba6db; no voters:
I20260812 21:03:53.639139 21477 raft_consensus.cc:2781] T c0b450c8fe3f45c190bacf1c14fa0d4a P 2b8213e6b65a43a7880293fbf5ce7f74 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 21:03:53.639197 21477 raft_consensus.cc:482] T c0b450c8fe3f45c190bacf1c14fa0d4a P 2b8213e6b65a43a7880293fbf5ce7f74 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 21:03:53.639226 21477 raft_consensus.cc:3037] T c0b450c8fe3f45c190bacf1c14fa0d4a P 2b8213e6b65a43a7880293fbf5ce7f74 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:03:53.639988 21477 raft_consensus.cc:504] T c0b450c8fe3f45c190bacf1c14fa0d4a P 2b8213e6b65a43a7880293fbf5ce7f74 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b8213e6b65a43a7880293fbf5ce7f74" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 43863 } } peers { permanent_uuid: "d756bc97f1a94cec9d79612302f2266a" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 37147 } } peers { permanent_uuid: "c3ded1c502a64427aae188ab512ba6db" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 36923 } }
I20260812 21:03:53.640117 21477 leader_election.cc:288] T c0b450c8fe3f45c190bacf1c14fa0d4a P 2b8213e6b65a43a7880293fbf5ce7f74 [CANDIDATE]: Term 1 election: Requested vote from peers d756bc97f1a94cec9d79612302f2266a (127.11.254.129:37147), c3ded1c502a64427aae188ab512ba6db (127.11.254.130:36923)
I20260812 21:03:53.640259 21137 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c0b450c8fe3f45c190bacf1c14fa0d4a" candidate_uuid: "2b8213e6b65a43a7880293fbf5ce7f74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3ded1c502a64427aae188ab512ba6db"
I20260812 21:03:53.640275 20907 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c0b450c8fe3f45c190bacf1c14fa0d4a" candidate_uuid: "2b8213e6b65a43a7880293fbf5ce7f74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d756bc97f1a94cec9d79612302f2266a"
I20260812 21:03:53.640333 21137 raft_consensus.cc:3037] T c0b450c8fe3f45c190bacf1c14fa0d4a P c3ded1c502a64427aae188ab512ba6db [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:03:53.640347 20907 raft_consensus.cc:3037] T c0b450c8fe3f45c190bacf1c14fa0d4a P d756bc97f1a94cec9d79612302f2266a [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:03:53.641049 21137 raft_consensus.cc:2454] T c0b450c8fe3f45c190bacf1c14fa0d4a P c3ded1c502a64427aae188ab512ba6db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b8213e6b65a43a7880293fbf5ce7f74 in term 1.
I20260812 21:03:53.641130 20907 raft_consensus.cc:2454] T c0b450c8fe3f45c190bacf1c14fa0d4a P d756bc97f1a94cec9d79612302f2266a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b8213e6b65a43a7880293fbf5ce7f74 in term 1.
I20260812 21:03:53.641193 21241 leader_election.cc:302] T c0b450c8fe3f45c190bacf1c14fa0d4a P 2b8213e6b65a43a7880293fbf5ce7f74 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b8213e6b65a43a7880293fbf5ce7f74, c3ded1c502a64427aae188ab512ba6db; no voters:
I20260812 21:03:53.641268 21477 raft_consensus.cc:2781] T c0b450c8fe3f45c190bacf1c14fa0d4a P 2b8213e6b65a43a7880293fbf5ce7f74 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:03:53.641410 21477 raft_consensus.cc:686] T c0b450c8fe3f45c190bacf1c14fa0d4a P 2b8213e6b65a43a7880293fbf5ce7f74 [term 1 LEADER]: Becoming Leader. State: Replica: 2b8213e6b65a43a7880293fbf5ce7f74, State: Running, Role: LEADER
I20260812 21:03:53.641496 21477 consensus_queue.cc:237] T c0b450c8fe3f45c190bacf1c14fa0d4a P 2b8213e6b65a43a7880293fbf5ce7f74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b8213e6b65a43a7880293fbf5ce7f74" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 43863 } } peers { permanent_uuid: "d756bc97f1a94cec9d79612302f2266a" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 37147 } } peers { permanent_uuid: "c3ded1c502a64427aae188ab512ba6db" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 36923 } }
I20260812 21:03:53.642314 20654 catalog_manager.cc:5495] T c0b450c8fe3f45c190bacf1c14fa0d4a P 2b8213e6b65a43a7880293fbf5ce7f74 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2b8213e6b65a43a7880293fbf5ce7f74 (127.11.254.131). New cstate: current_term: 1 leader_uuid: "2b8213e6b65a43a7880293fbf5ce7f74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b8213e6b65a43a7880293fbf5ce7f74" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 43863 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d756bc97f1a94cec9d79612302f2266a" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 37147 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c3ded1c502a64427aae188ab512ba6db" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 36923 } health_report { overall_health: UNKNOWN } } }
I20260812 21:03:53.678119 21459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:53.677190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33074 (local address 127.11.254.131:43863)
0812 21:03:53.677247 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0812 21:03:53.677249 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:03:53.677257 (+ 8us) server_negotiation.cc:407] Connection header received
0812 21:03:53.677293 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:03:53.677294 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:03:53.677307 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:03:53.677349 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:03:53.677882 (+ 533us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:03:53.677883 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:03:53.677883 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 21:03:53.677885 (+ 2us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:03:53.677895 (+ 10us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 21:03:53.677898 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:03:53.677932 (+ 34us) server_negotiation.cc:1035] Waiting for connection context
0812 21:03:53.678013 (+ 81us) server_negotiation.cc:299] Negotiation successful
0812 21:03:53.678025 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260812 21:03:53.679956 21137 raft_consensus.cc:1261] T c0b450c8fe3f45c190bacf1c14fa0d4a P c3ded1c502a64427aae188ab512ba6db [term 1 FOLLOWER]: Refusing update from remote peer 2b8213e6b65a43a7880293fbf5ce7f74: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 21:03:53.679963 20907 raft_consensus.cc:1261] T c0b450c8fe3f45c190bacf1c14fa0d4a P d756bc97f1a94cec9d79612302f2266a [term 1 FOLLOWER]: Refusing update from remote peer 2b8213e6b65a43a7880293fbf5ce7f74: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 21:03:53.680302 21477 consensus_queue.cc:1035] T c0b450c8fe3f45c190bacf1c14fa0d4a P 2b8213e6b65a43a7880293fbf5ce7f74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c3ded1c502a64427aae188ab512ba6db" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 36923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 21:03:53.680460 21482 consensus_queue.cc:1035] T c0b450c8fe3f45c190bacf1c14fa0d4a P 2b8213e6b65a43a7880293fbf5ce7f74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d756bc97f1a94cec9d79612302f2266a" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 37147 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 21:03:53.683328 21495 mvcc.cc:205] Tried to move back new op lower bound from 7317785123550285824 to 7317785123396247552. Current Snapshot: MvccSnapshot[applied={T|T < 7317785123550285824}]
I20260812 21:03:53.683604 21497 mvcc.cc:205] Tried to move back new op lower bound from 7317785123550285824 to 7317785123396247552. Current Snapshot: MvccSnapshot[applied={T|T < 7317785123550285824}]
I20260812 21:03:53.684121 21493 mvcc.cc:205] Tried to move back new op lower bound from 7317785123550285824 to 7317785123396247552. Current Snapshot: MvccSnapshot[applied={T|T < 7317785123550285824 or (T in {7317785123550285824})}]
I20260812 21:03:53.685482 20654 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:42428:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 21:03:53.685575 20654 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:42428:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 21:03:53.686308 20654 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d0335d0f174540c982708fdf9c2abd22: Sending DeleteTablet for 3 replicas of tablet c0b450c8fe3f45c190bacf1c14fa0d4a
I20260812 21:03:53.686661 21315 tablet_service.cc:1514] Processing DeleteTablet for tablet c0b450c8fe3f45c190bacf1c14fa0d4a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:03:53 UTC) from {username='slave'} at 127.0.0.1:33072
I20260812 21:03:53.686668 20882 tablet_service.cc:1514] Processing DeleteTablet for tablet c0b450c8fe3f45c190bacf1c14fa0d4a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:03:53 UTC) from {username='slave'} at 127.0.0.1:46000
I20260812 21:03:53.686888 21511 tablet_replica.cc:330] T c0b450c8fe3f45c190bacf1c14fa0d4a P 2b8213e6b65a43a7880293fbf5ce7f74: stopping tablet replica
I20260812 21:03:53.686908 21512 tablet_replica.cc:330] T c0b450c8fe3f45c190bacf1c14fa0d4a P d756bc97f1a94cec9d79612302f2266a: stopping tablet replica
I20260812 21:03:53.686980 21511 raft_consensus.cc:2229] T c0b450c8fe3f45c190bacf1c14fa0d4a P 2b8213e6b65a43a7880293fbf5ce7f74 [term 1 LEADER]: Raft consensus shutting down.
I20260812 21:03:53.686988 21512 raft_consensus.cc:2229] T c0b450c8fe3f45c190bacf1c14fa0d4a P d756bc97f1a94cec9d79612302f2266a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 21:03:53.686992 21098 tablet_service.cc:1514] Processing DeleteTablet for tablet c0b450c8fe3f45c190bacf1c14fa0d4a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:03:53 UTC) from {username='slave'} at 127.0.0.1:57990
I20260812 21:03:53.687068 21511 raft_consensus.cc:2258] T c0b450c8fe3f45c190bacf1c14fa0d4a P 2b8213e6b65a43a7880293fbf5ce7f74 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 21:03:53.687098 21512 raft_consensus.cc:2258] T c0b450c8fe3f45c190bacf1c14fa0d4a P d756bc97f1a94cec9d79612302f2266a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 21:03:53.687516 21511 ts_tablet_manager.cc:1905] T c0b450c8fe3f45c190bacf1c14fa0d4a P 2b8213e6b65a43a7880293fbf5ce7f74: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 21:03:53.687507 21512 ts_tablet_manager.cc:1905] T c0b450c8fe3f45c190bacf1c14fa0d4a P d756bc97f1a94cec9d79612302f2266a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 21:03:53.687672 21516 tablet_replica.cc:330] T c0b450c8fe3f45c190bacf1c14fa0d4a P c3ded1c502a64427aae188ab512ba6db: stopping tablet replica
I20260812 21:03:53.687731 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 20771
I20260812 21:03:53.687860 21516 raft_consensus.cc:2229] T c0b450c8fe3f45c190bacf1c14fa0d4a P c3ded1c502a64427aae188ab512ba6db [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 21:03:53.688014 21516 raft_consensus.cc:2258] T c0b450c8fe3f45c190bacf1c14fa0d4a P c3ded1c502a64427aae188ab512ba6db [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 21:03:53.689690 21511 ts_tablet_manager.cc:1918] T c0b450c8fe3f45c190bacf1c14fa0d4a P 2b8213e6b65a43a7880293fbf5ce7f74: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 21:03:53.689759 21511 log.cc:1196] T c0b450c8fe3f45c190bacf1c14fa0d4a P 2b8213e6b65a43a7880293fbf5ce7f74: Deleting WAL directory at /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786568607354921-12282-0/minicluster-data/ts-2/wal/wals/c0b450c8fe3f45c190bacf1c14fa0d4a
I20260812 21:03:53.690078 21511 ts_tablet_manager.cc:1939] T c0b450c8fe3f45c190bacf1c14fa0d4a P 2b8213e6b65a43a7880293fbf5ce7f74: Deleting consensus metadata
I20260812 21:03:53.690460 20630 catalog_manager.cc:4841] TS 2b8213e6b65a43a7880293fbf5ce7f74 (127.11.254.131:43863): tablet c0b450c8fe3f45c190bacf1c14fa0d4a (table test-table [id=e8f75efb3d9344d68e8b47d1c796d2d4]) successfully deleted
I20260812 21:03:53.695446 21516 ts_tablet_manager.cc:1905] T c0b450c8fe3f45c190bacf1c14fa0d4a P c3ded1c502a64427aae188ab512ba6db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 21:03:53.697412 21516 ts_tablet_manager.cc:1918] T c0b450c8fe3f45c190bacf1c14fa0d4a P c3ded1c502a64427aae188ab512ba6db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 21:03:53.697507 21516 log.cc:1196] T c0b450c8fe3f45c190bacf1c14fa0d4a P c3ded1c502a64427aae188ab512ba6db: Deleting WAL directory at /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786568607354921-12282-0/minicluster-data/ts-1/wal/wals/c0b450c8fe3f45c190bacf1c14fa0d4a
I20260812 21:03:53.697834 21516 ts_tablet_manager.cc:1939] T c0b450c8fe3f45c190bacf1c14fa0d4a P c3ded1c502a64427aae188ab512ba6db: Deleting consensus metadata
I20260812 21:03:53.698431 20630 catalog_manager.cc:4841] TS c3ded1c502a64427aae188ab512ba6db (127.11.254.130:36923): tablet c0b450c8fe3f45c190bacf1c14fa0d4a (table test-table [id=e8f75efb3d9344d68e8b47d1c796d2d4]) successfully deleted
W20260812 21:03:53.704082 20631 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.254.129:37147 (error 108)
I20260812 21:03:53.704285 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 20997
I20260812 21:03:53.704664 21461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:53.704537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.129:37147 (local address 127.0.0.1:46004)
0812 21:03:53.704588 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0812 21:03:53.704627 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.129:37147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260812 21:03:53.704771 20631 catalog_manager.cc:4584] TS d756bc97f1a94cec9d79612302f2266a (127.11.254.129:37147): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c0b450c8fe3f45c190bacf1c14fa0d4a: Network error: Client connection negotiation failed: client connection to 127.11.254.129:37147: connect: Connection refused (error 111)
I20260812 21:03:53.719065 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 21218
I20260812 21:03:53.732519 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 20608
2026-08-12T21:03:53Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (809 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260812 21:03:53.756127 12282 mini_oidc.cc:152] Starting JWKS server
I20260812 21:03:53.757349 12282 webserver.cc:460] Webserver started at https://127.0.0.1:45559/ using document root <none> and password file <none>
I20260812 21:03:53.757443 12282 mini_oidc.cc:189] Starting OIDC Discovery server
I20260812 21:03:53.757615 12282 webserver.cc:460] Webserver started at http://127.0.0.1:42059/ using document root <none> and password file <none>
2026-08-12T21:03:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T21:03:53Z Disabled control of system clock
I20260812 21:03:53.766096 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786568607354921-12282-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786568607354921-12282-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786568607354921-12282-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.254.190:46567
--webserver_interface=127.11.254.190
--webserver_port=0
--builtin_ntp_servers=127.11.254.148:33689
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.254.190:46567
--enable_jwt_token_auth=true
--jwks_url=https://localhost:45559/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786568607354921-12282-0/testchainca.cert with env {}
W20260812 21:03:53.847906 21524 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:03:53.848129 21524 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:03:53.848151 21524 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:03:53.849306 21524 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 21:03:53.849345 21524 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:03:53.849357 21524 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:03:53.849367 21524 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 21:03:53.849378 21524 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260812 21:03:53.849390 21524 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260812 21:03:53.849399 21524 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:45559/jwks.json
I20260812 21:03:53.851258 21524 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.254.148:33689
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786568607354921-12282-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786568607354921-12282-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.254.190:46567
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.254.190:46567
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:45559/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.254.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786568607354921-12282-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21524
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786568607354921-12282-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:03:53.851500 21524 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:03:53.851688 21524 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:03:53.856746 21535 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:03:53.856806 21524 server_base.cc:1034] running on GCE node
W20260812 21:03:53.856767 21529 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:03:53.856762 21530 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:03:53.857286 21524 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:03:53.857563 21524 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:03:53.858693 21524 hybrid_clock.cc:648] HybridClock initialized: now 1786568633858680 us; error 28 us; skew 500 ppm
I20260812 21:03:53.859916 21524 webserver.cc:460] Webserver started at http://127.11.254.190:45173/ using document root <none> and password file <none>
I20260812 21:03:53.860188 21524 fs_manager.cc:359] Metadata directory not provided
I20260812 21:03:53.860236 21524 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:03:53.860359 21524 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:03:53.861428 21524 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786568607354921-12282-0/minicluster-data/master-0/data/instance:
uuid: "06b2926bd6434e2095b54f13a7d836b2"
format_stamp: "Formatted at 2026-08-12 21:03:53 on dist-test-slave-vpvm"
server_key: "79ff5ba2d2b8757aea7e6e6791b343a2"
server_key_iv: "4fef08773a86bd215288277749bcb088"
server_key_version: "encryptionkey@0"
I20260812 21:03:53.861765 21524 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786568607354921-12282-0/minicluster-data/master-0/wal/instance:
uuid: "06b2926bd6434e2095b54f13a7d836b2"
format_stamp: "Formatted at 2026-08-12 21:03:53 on dist-test-slave-vpvm"
server_key: "79ff5ba2d2b8757aea7e6e6791b343a2"
server_key_iv: "4fef08773a86bd215288277749bcb088"
server_key_version: "encryptionkey@0"
I20260812 21:03:53.863085 21524 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 21:03:53.863786 21548 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:03:53.863930 21524 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 21:03:53.864019 21524 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786568607354921-12282-0/minicluster-data/master-0/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786568607354921-12282-0/minicluster-data/master-0/wal
uuid: "06b2926bd6434e2095b54f13a7d836b2"
format_stamp: "Formatted at 2026-08-12 21:03:53 on dist-test-slave-vpvm"
server_key: "79ff5ba2d2b8757aea7e6e6791b343a2"
server_key_iv: "4fef08773a86bd215288277749bcb088"
server_key_version: "encryptionkey@0"
I20260812 21:03:53.864084 21524 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786568607354921-12282-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786568607354921-12282-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786568607354921-12282-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:03:53.871174 21524 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:03:53.871439 21524 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:03:53.871551 21524 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:03:53.875480 21524 rpc_server.cc:307] RPC server started. Bound to: 127.11.254.190:46567
I20260812 21:03:53.875546 21636 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.254.190:46567 every 8 connection(s)
I20260812 21:03:53.875880 21524 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
I20260812 21:03:53.877009 21640 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:03:53.879472 21640 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 06b2926bd6434e2095b54f13a7d836b2: Bootstrap starting.
I20260812 21:03:53.880128 21640 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 06b2926bd6434e2095b54f13a7d836b2: Neither blocks nor log segments found. Creating new log.
I20260812 21:03:53.880429 21640 log.cc:824] T 00000000000000000000000000000000 P 06b2926bd6434e2095b54f13a7d836b2: Log is configured to *not* fsync() on all Append() calls
I20260812 21:03:53.880640 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 21524
I20260812 21:03:53.880738 12282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786568607354921-12282-0/minicluster-data/master-0/wal/instance
I20260812 21:03:53.880911 12282 external_mini_cluster.cc:1428] Setting key 53d57188f8925f50c054444dbb996988
I20260812 21:03:53.881198 21640 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 06b2926bd6434e2095b54f13a7d836b2: No bootstrap required, opened a new log
I20260812 21:03:53.883986 21640 raft_consensus.cc:348] T 00000000000000000000000000000000 P 06b2926bd6434e2095b54f13a7d836b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06b2926bd6434e2095b54f13a7d836b2" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 46567 } }
I20260812 21:03:53.884176 21640 raft_consensus.cc:374] T 00000000000000000000000000000000 P 06b2926bd6434e2095b54f13a7d836b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:03:53.884208 21640 raft_consensus.cc:729] T 00000000000000000000000000000000 P 06b2926bd6434e2095b54f13a7d836b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06b2926bd6434e2095b54f13a7d836b2, State: Initialized, Role: FOLLOWER
I20260812 21:03:53.884328 21640 consensus_queue.cc:260] T 00000000000000000000000000000000 P 06b2926bd6434e2095b54f13a7d836b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06b2926bd6434e2095b54f13a7d836b2" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 46567 } }
I20260812 21:03:53.884402 21640 raft_consensus.cc:388] T 00000000000000000000000000000000 P 06b2926bd6434e2095b54f13a7d836b2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 21:03:53.884434 21640 raft_consensus.cc:482] T 00000000000000000000000000000000 P 06b2926bd6434e2095b54f13a7d836b2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 21:03:53.884476 21640 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 06b2926bd6434e2095b54f13a7d836b2 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:03:53.885058 21640 raft_consensus.cc:504] T 00000000000000000000000000000000 P 06b2926bd6434e2095b54f13a7d836b2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06b2926bd6434e2095b54f13a7d836b2" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 46567 } }
I20260812 21:03:53.885159 21640 leader_election.cc:302] T 00000000000000000000000000000000 P 06b2926bd6434e2095b54f13a7d836b2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 06b2926bd6434e2095b54f13a7d836b2; no voters:
I20260812 21:03:53.885331 21640 leader_election.cc:288] T 00000000000000000000000000000000 P 06b2926bd6434e2095b54f13a7d836b2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 21:03:53.885391 21645 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 06b2926bd6434e2095b54f13a7d836b2 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:03:53.885547 21645 raft_consensus.cc:686] T 00000000000000000000000000000000 P 06b2926bd6434e2095b54f13a7d836b2 [term 1 LEADER]: Becoming Leader. State: Replica: 06b2926bd6434e2095b54f13a7d836b2, State: Running, Role: LEADER
I20260812 21:03:53.885601 21640 sys_catalog.cc:564] T 00000000000000000000000000000000 P 06b2926bd6434e2095b54f13a7d836b2 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 21:03:53.885648 21645 consensus_queue.cc:237] T 00000000000000000000000000000000 P 06b2926bd6434e2095b54f13a7d836b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06b2926bd6434e2095b54f13a7d836b2" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 46567 } }
I20260812 21:03:53.886059 21647 sys_catalog.cc:455] T 00000000000000000000000000000000 P 06b2926bd6434e2095b54f13a7d836b2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "06b2926bd6434e2095b54f13a7d836b2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06b2926bd6434e2095b54f13a7d836b2" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 46567 } } }
I20260812 21:03:53.886153 21647 sys_catalog.cc:458] T 00000000000000000000000000000000 P 06b2926bd6434e2095b54f13a7d836b2 [sys.catalog]: This master's current role is: LEADER
I20260812 21:03:53.886339 21649 sys_catalog.cc:455] T 00000000000000000000000000000000 P 06b2926bd6434e2095b54f13a7d836b2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 06b2926bd6434e2095b54f13a7d836b2. Latest consensus state: current_term: 1 leader_uuid: "06b2926bd6434e2095b54f13a7d836b2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06b2926bd6434e2095b54f13a7d836b2" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 46567 } } }
I20260812 21:03:53.886410 21649 sys_catalog.cc:458] T 00000000000000000000000000000000 P 06b2926bd6434e2095b54f13a7d836b2 [sys.catalog]: This master's current role is: LEADER
I20260812 21:03:53.886646 21657 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 21:03:53.887164 21657 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 21:03:53.887733 21644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:53.881438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54228 (local address 127.11.254.190:46567)
0812 21:03:53.881683 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0812 21:03:53.881692 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 21:03:53.881728 (+ 36us) server_negotiation.cc:407] Connection header received
0812 21:03:53.882538 (+ 810us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:03:53.882548 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:03:53.882630 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:03:53.882784 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:03:53.883002 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:53.884374 (+ 1372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:53.885250 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:53.885877 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:53.886408 (+ 531us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:53.886781 (+ 373us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:03:53.886785 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:03:53.886785 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 21:03:53.887023 (+ 238us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:03:53.887070 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 21:03:53.887086 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:03:53.887382 (+ 296us) server_negotiation.cc:1035] Waiting for connection context
0812 21:03:53.887420 (+ 38us) server_negotiation.cc:299] Negotiation successful
0812 21:03:53.887511 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":99,"threads_started":1}
I20260812 21:03:53.888780 12282 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 21:03:53.888850 21657 catalog_manager.cc:1347] Generated new cluster ID: ef3401b6330041b89c693b131d8c842c
I20260812 21:03:53.888903 21657 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 21:03:53.898557 21657 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 21:03:53.899166 21657 catalog_manager.cc:1504] Loading token signing keys...
I20260812 21:03:53.903218 21657 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 06b2926bd6434e2095b54f13a7d836b2: Generated new TSK 0
I20260812 21:03:53.903401 21657 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260812 21:03:53.928102 21681 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260812 21:03:53.928244 21681 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
W20260812 21:03:53.928911 21679 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:03:53.912132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:46567 (local address 127.0.0.1:54230)
0812 21:03:53.912348 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0812 21:03:53.912358 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 21:03:53.912424 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:03:53.912648 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:03:53.912649 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:03:53.912658 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0812 21:03:53.912726 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:53.912728 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:53.913682 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:53.913682 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:53.914364 (+ 682us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:53.914368 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:53.914815 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:53.914816 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:53.914862 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:53.914886 (+ 24us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0812 21:03:53.928362 (+ 13476us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45559/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0812 21:03:53.928478 (+ 116us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.11.254.190:46567: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45559/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":101,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.11.254.190:46567: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45559/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260812 21:03:53.929250 21644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:53.912229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54230 (local address 127.11.254.190:46567)
0812 21:03:53.912275 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0812 21:03:53.912277 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:03:53.912414 (+ 137us) server_negotiation.cc:407] Connection header received
0812 21:03:53.912504 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:03:53.912505 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:03:53.912520 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:03:53.912601 (+ 81us) server_negotiation.cc:226] Negotiated authn=JWT
0812 21:03:53.912792 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:53.913602 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:53.914437 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:53.914736 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:53.914805 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:53.914990 (+ 185us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0812 21:03:53.928186 (+ 13196us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45559/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0812 21:03:53.929172 (+ 986us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:54230: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45559/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 21:03:53.929279 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 21524
W20260812 21:03:53.929319 21644 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:54230: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45559/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
2026-08-12T21:03:53Z chronyd exiting
I20260812 21:03:53.989287 12282 test_util.cc:175] -----------------------------------------------
I20260812 21:03:53.989426 12282 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786568607354921-12282-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (234 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260812 21:03:53.990295 12282 mini_oidc.cc:152] Starting JWKS server
I20260812 21:03:53.990993 12282 webserver.cc:460] Webserver started at https://127.0.0.1:43033/ using document root <none> and password file <none>
I20260812 21:03:53.991036 12282 mini_oidc.cc:189] Starting OIDC Discovery server
I20260812 21:03:53.991160 12282 webserver.cc:460] Webserver started at http://127.0.0.1:42415/ using document root <none> and password file <none>
2026-08-12T21:03:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T21:03:53Z Disabled control of system clock
I20260812 21:03:53.999903 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786568607354921-12282-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786568607354921-12282-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786568607354921-12282-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.254.190:44141
--webserver_interface=127.11.254.190
--webserver_port=0
--builtin_ntp_servers=127.11.254.148:34289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.254.190:44141
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43033/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786568607354921-12282-0/testchainca.cert with env {}
W20260812 21:03:54.081794 21689 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:03:54.082047 21689 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:03:54.082079 21689 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:03:54.083303 21689 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 21:03:54.083348 21689 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:03:54.083364 21689 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:03:54.083379 21689 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 21:03:54.083395 21689 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260812 21:03:54.083410 21689 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260812 21:03:54.083432 21689 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:43033/jwks.json
I20260812 21:03:54.085012 21689 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.254.148:34289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786568607354921-12282-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786568607354921-12282-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.254.190:44141
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.254.190:44141
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43033/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.254.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786568607354921-12282-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21689
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786568607354921-12282-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:03:54.085278 21689 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:03:54.085482 21689 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:03:54.090265 21696 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:03:54.090241 21697 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:03:54.090283 21702 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:03:54.090694 21689 server_base.cc:1034] running on GCE node
I20260812 21:03:54.090947 21689 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:03:54.091244 21689 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:03:54.092402 21689 hybrid_clock.cc:648] HybridClock initialized: now 1786568634092386 us; error 28 us; skew 500 ppm
I20260812 21:03:54.093779 21689 webserver.cc:460] Webserver started at http://127.11.254.190:33389/ using document root <none> and password file <none>
I20260812 21:03:54.094050 21689 fs_manager.cc:359] Metadata directory not provided
I20260812 21:03:54.094099 21689 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:03:54.094223 21689 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:03:54.095296 21689 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786568607354921-12282-0/minicluster-data/master-0/data/instance:
uuid: "6f47ee3625974e009e8d503ab94b72b5"
format_stamp: "Formatted at 2026-08-12 21:03:54 on dist-test-slave-vpvm"
server_key: "7f45104901637dc4cbf0011668b3fce6"
server_key_iv: "a2881eb8e32245fdd09784b9e5785477"
server_key_version: "encryptionkey@0"
I20260812 21:03:54.095721 21689 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786568607354921-12282-0/minicluster-data/master-0/wal/instance:
uuid: "6f47ee3625974e009e8d503ab94b72b5"
format_stamp: "Formatted at 2026-08-12 21:03:54 on dist-test-slave-vpvm"
server_key: "7f45104901637dc4cbf0011668b3fce6"
server_key_iv: "a2881eb8e32245fdd09784b9e5785477"
server_key_version: "encryptionkey@0"
I20260812 21:03:54.096982 21689 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 21:03:54.097666 21717 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:03:54.097841 21689 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 21:03:54.097925 21689 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786568607354921-12282-0/minicluster-data/master-0/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786568607354921-12282-0/minicluster-data/master-0/wal
uuid: "6f47ee3625974e009e8d503ab94b72b5"
format_stamp: "Formatted at 2026-08-12 21:03:54 on dist-test-slave-vpvm"
server_key: "7f45104901637dc4cbf0011668b3fce6"
server_key_iv: "a2881eb8e32245fdd09784b9e5785477"
server_key_version: "encryptionkey@0"
I20260812 21:03:54.098026 21689 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786568607354921-12282-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786568607354921-12282-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786568607354921-12282-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:03:54.108803 21689 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:03:54.109102 21689 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:03:54.109229 21689 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:03:54.113145 21689 rpc_server.cc:307] RPC server started. Bound to: 127.11.254.190:44141
I20260812 21:03:54.113179 21811 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.254.190:44141 every 8 connection(s)
I20260812 21:03:54.113504 21689 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
I20260812 21:03:54.114228 21812 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:03:54.114359 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 21689
I20260812 21:03:54.114455 12282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786568607354921-12282-0/minicluster-data/master-0/wal/instance
I20260812 21:03:54.114639 12282 external_mini_cluster.cc:1428] Setting key 556f3a632b4957eee1da2b3c4299d6cc
I20260812 21:03:54.118289 21812 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f47ee3625974e009e8d503ab94b72b5: Bootstrap starting.
I20260812 21:03:54.118953 21812 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6f47ee3625974e009e8d503ab94b72b5: Neither blocks nor log segments found. Creating new log.
I20260812 21:03:54.119264 21812 log.cc:824] T 00000000000000000000000000000000 P 6f47ee3625974e009e8d503ab94b72b5: Log is configured to *not* fsync() on all Append() calls
I20260812 21:03:54.119959 21812 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f47ee3625974e009e8d503ab94b72b5: No bootstrap required, opened a new log
I20260812 21:03:54.120664 21815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:54.115137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54664 (local address 127.11.254.190:44141)
0812 21:03:54.115352 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0812 21:03:54.115361 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 21:03:54.115394 (+ 33us) server_negotiation.cc:407] Connection header received
0812 21:03:54.115955 (+ 561us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:03:54.115965 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:03:54.116048 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:03:54.116220 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:03:54.116389 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:54.117710 (+ 1321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:54.118568 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:54.119175 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:54.119300 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:54.119953 (+ 653us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:03:54.119958 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:03:54.119958 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 21:03:54.120165 (+ 207us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:03:54.120207 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 21:03:54.120221 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:03:54.120289 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0812 21:03:54.120347 (+ 58us) server_negotiation.cc:299] Negotiation successful
0812 21:03:54.120434 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":82,"threads_started":1}
I20260812 21:03:54.121798 12282 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 21:03:54.122200 21812 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6f47ee3625974e009e8d503ab94b72b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f47ee3625974e009e8d503ab94b72b5" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 44141 } }
I20260812 21:03:54.122340 21812 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6f47ee3625974e009e8d503ab94b72b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:03:54.122362 21812 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6f47ee3625974e009e8d503ab94b72b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f47ee3625974e009e8d503ab94b72b5, State: Initialized, Role: FOLLOWER
I20260812 21:03:54.122462 21812 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6f47ee3625974e009e8d503ab94b72b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f47ee3625974e009e8d503ab94b72b5" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 44141 } }
I20260812 21:03:54.122545 21812 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6f47ee3625974e009e8d503ab94b72b5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 21:03:54.122584 21812 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6f47ee3625974e009e8d503ab94b72b5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 21:03:54.122632 21812 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6f47ee3625974e009e8d503ab94b72b5 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:03:54.123267 21812 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6f47ee3625974e009e8d503ab94b72b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f47ee3625974e009e8d503ab94b72b5" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 44141 } }
I20260812 21:03:54.123369 21812 leader_election.cc:302] T 00000000000000000000000000000000 P 6f47ee3625974e009e8d503ab94b72b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6f47ee3625974e009e8d503ab94b72b5; no voters:
I20260812 21:03:54.123526 21812 leader_election.cc:288] T 00000000000000000000000000000000 P 6f47ee3625974e009e8d503ab94b72b5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 21:03:54.123646 21819 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6f47ee3625974e009e8d503ab94b72b5 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:03:54.123786 21819 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6f47ee3625974e009e8d503ab94b72b5 [term 1 LEADER]: Becoming Leader. State: Replica: 6f47ee3625974e009e8d503ab94b72b5, State: Running, Role: LEADER
I20260812 21:03:54.123811 21812 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6f47ee3625974e009e8d503ab94b72b5 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 21:03:54.123911 21819 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6f47ee3625974e009e8d503ab94b72b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f47ee3625974e009e8d503ab94b72b5" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 44141 } }
I20260812 21:03:54.124279 21821 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f47ee3625974e009e8d503ab94b72b5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6f47ee3625974e009e8d503ab94b72b5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f47ee3625974e009e8d503ab94b72b5" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 44141 } } }
I20260812 21:03:54.124365 21821 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f47ee3625974e009e8d503ab94b72b5 [sys.catalog]: This master's current role is: LEADER
I20260812 21:03:54.124526 21824 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f47ee3625974e009e8d503ab94b72b5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6f47ee3625974e009e8d503ab94b72b5. Latest consensus state: current_term: 1 leader_uuid: "6f47ee3625974e009e8d503ab94b72b5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f47ee3625974e009e8d503ab94b72b5" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 44141 } } }
I20260812 21:03:54.124611 21824 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f47ee3625974e009e8d503ab94b72b5 [sys.catalog]: This master's current role is: LEADER
I20260812 21:03:54.124681 21838 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 21:03:54.125187 21838 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 21:03:54.126679 21838 catalog_manager.cc:1347] Generated new cluster ID: 87910593a5d94b38b87b50095a8e8f9e
I20260812 21:03:54.126735 21838 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 21:03:54.132040 21838 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 21:03:54.132525 21838 catalog_manager.cc:1504] Loading token signing keys...
I20260812 21:03:54.138077 21838 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6f47ee3625974e009e8d503ab94b72b5: Generated new TSK 0
I20260812 21:03:54.138250 21838 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260812 21:03:54.166692 21864 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260812 21:03:54.166791 21864 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260812 21:03:54.167063 21815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:54.145210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54674 (local address 127.11.254.190:44141)
0812 21:03:54.145260 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0812 21:03:54.145262 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:03:54.145360 (+ 98us) server_negotiation.cc:407] Connection header received
0812 21:03:54.145431 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:03:54.145431 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:03:54.145444 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:03:54.145504 (+ 60us) server_negotiation.cc:226] Negotiated authn=JWT
0812 21:03:54.145647 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:54.146197 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:54.146968 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:54.147165 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:54.147223 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:54.147390 (+ 167us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0812 21:03:54.166832 (+ 19442us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43033/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0812 21:03:54.167004 (+ 172us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:54674: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43033/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":26}
W20260812 21:03:54.167120 21815 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:54674: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43033/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260812 21:03:54.167188 21863 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:03:54.145145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:44141 (local address 127.0.0.1:54674)
0812 21:03:54.145293 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0812 21:03:54.145303 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 21:03:54.145371 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:03:54.145507 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:03:54.145508 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:03:54.145510 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0812 21:03:54.145584 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:54.145587 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:54.146282 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:54.146284 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:54.146893 (+ 609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:54.146895 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:54.147232 (+ 337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:54.147234 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:54.147298 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:54.147303 (+ 5us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0812 21:03:54.166971 (+ 19668us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43033/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0812 21:03:54.167088 (+ 117us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.11.254.190:44141: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43033/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":85,"threads_started":1}
I20260812 21:03:54.167493 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 21689
2026-08-12T21:03:54Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (224 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260812 21:03:54.214887 12282 mini_oidc.cc:152] Starting JWKS server
I20260812 21:03:54.215583 12282 webserver.cc:460] Webserver started at https://127.0.0.1:34641/ using document root <none> and password file <none>
I20260812 21:03:54.215631 12282 mini_oidc.cc:189] Starting OIDC Discovery server
I20260812 21:03:54.215768 12282 webserver.cc:460] Webserver started at http://127.0.0.1:34169/ using document root <none> and password file <none>
2026-08-12T21:03:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T21:03:54Z Disabled control of system clock
I20260812 21:03:54.224447 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786568607354921-12282-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786568607354921-12282-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786568607354921-12282-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.254.190:39283
--webserver_interface=127.11.254.190
--webserver_port=0
--builtin_ntp_servers=127.11.254.148:43651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.254.190:39283
--enable_jwt_token_auth=true
--jwks_url=https://localhost:34641/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260812 21:03:54.309170 21875 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:03:54.309381 21875 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:03:54.309402 21875 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:03:54.310606 21875 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 21:03:54.310645 21875 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:03:54.310657 21875 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:03:54.310668 21875 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 21:03:54.310678 21875 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260812 21:03:54.310689 21875 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260812 21:03:54.310707 21875 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:34641/jwks.json
I20260812 21:03:54.312049 21875 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.254.148:43651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786568607354921-12282-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786568607354921-12282-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.254.190:39283
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.254.190:39283
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:34641/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.254.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21875
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786568607354921-12282-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:03:54.312252 21875 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:03:54.312503 21875 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:03:54.317377 21893 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:03:54.317364 21887 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:03:54.317466 21875 server_base.cc:1034] running on GCE node
W20260812 21:03:54.317369 21885 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:03:54.317819 21875 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:03:54.318116 21875 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:03:54.319250 21875 hybrid_clock.cc:648] HybridClock initialized: now 1786568634319236 us; error 32 us; skew 500 ppm
I20260812 21:03:54.320466 21875 webserver.cc:460] Webserver started at http://127.11.254.190:35107/ using document root <none> and password file <none>
I20260812 21:03:54.320734 21875 fs_manager.cc:359] Metadata directory not provided
I20260812 21:03:54.320781 21875 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:03:54.320905 21875 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:03:54.322017 21875 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786568607354921-12282-0/minicluster-data/master-0/data/instance:
uuid: "f3e09e0d3a6842a7a2994f8df349ea12"
format_stamp: "Formatted at 2026-08-12 21:03:54 on dist-test-slave-vpvm"
server_key: "0699019d23459eeee7748105e6c3cfce"
server_key_iv: "df947e45a2f20ab2a913faf60c0436bb"
server_key_version: "encryptionkey@0"
I20260812 21:03:54.322358 21875 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786568607354921-12282-0/minicluster-data/master-0/wal/instance:
uuid: "f3e09e0d3a6842a7a2994f8df349ea12"
format_stamp: "Formatted at 2026-08-12 21:03:54 on dist-test-slave-vpvm"
server_key: "0699019d23459eeee7748105e6c3cfce"
server_key_iv: "df947e45a2f20ab2a913faf60c0436bb"
server_key_version: "encryptionkey@0"
I20260812 21:03:54.323658 21875 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 21:03:54.324402 21903 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:03:54.324550 21875 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 21:03:54.324645 21875 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786568607354921-12282-0/minicluster-data/master-0/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786568607354921-12282-0/minicluster-data/master-0/wal
uuid: "f3e09e0d3a6842a7a2994f8df349ea12"
format_stamp: "Formatted at 2026-08-12 21:03:54 on dist-test-slave-vpvm"
server_key: "0699019d23459eeee7748105e6c3cfce"
server_key_iv: "df947e45a2f20ab2a913faf60c0436bb"
server_key_version: "encryptionkey@0"
I20260812 21:03:54.324709 21875 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786568607354921-12282-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786568607354921-12282-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786568607354921-12282-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:03:54.357132 21875 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:03:54.357507 21875 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:03:54.357652 21875 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:03:54.361760 21875 rpc_server.cc:307] RPC server started. Bound to: 127.11.254.190:39283
I20260812 21:03:54.361796 21997 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.254.190:39283 every 8 connection(s)
I20260812 21:03:54.362155 21875 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
I20260812 21:03:54.362849 21999 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:03:54.365691 21999 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f3e09e0d3a6842a7a2994f8df349ea12: Bootstrap starting.
I20260812 21:03:54.366307 21999 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f3e09e0d3a6842a7a2994f8df349ea12: Neither blocks nor log segments found. Creating new log.
I20260812 21:03:54.366590 21999 log.cc:824] T 00000000000000000000000000000000 P f3e09e0d3a6842a7a2994f8df349ea12: Log is configured to *not* fsync() on all Append() calls
I20260812 21:03:54.367250 21999 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f3e09e0d3a6842a7a2994f8df349ea12: No bootstrap required, opened a new log
I20260812 21:03:54.368880 21999 raft_consensus.cc:348] T 00000000000000000000000000000000 P f3e09e0d3a6842a7a2994f8df349ea12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3e09e0d3a6842a7a2994f8df349ea12" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 39283 } }
I20260812 21:03:54.369033 21999 raft_consensus.cc:374] T 00000000000000000000000000000000 P f3e09e0d3a6842a7a2994f8df349ea12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:03:54.369055 21999 raft_consensus.cc:729] T 00000000000000000000000000000000 P f3e09e0d3a6842a7a2994f8df349ea12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3e09e0d3a6842a7a2994f8df349ea12, State: Initialized, Role: FOLLOWER
I20260812 21:03:54.369208 21999 consensus_queue.cc:260] T 00000000000000000000000000000000 P f3e09e0d3a6842a7a2994f8df349ea12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3e09e0d3a6842a7a2994f8df349ea12" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 39283 } }
I20260812 21:03:54.369268 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 21875
I20260812 21:03:54.369292 21999 raft_consensus.cc:388] T 00000000000000000000000000000000 P f3e09e0d3a6842a7a2994f8df349ea12 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 21:03:54.369325 21999 raft_consensus.cc:482] T 00000000000000000000000000000000 P f3e09e0d3a6842a7a2994f8df349ea12 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 21:03:54.369340 12282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786568607354921-12282-0/minicluster-data/master-0/wal/instance
I20260812 21:03:54.369375 21999 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f3e09e0d3a6842a7a2994f8df349ea12 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:03:54.369519 12282 external_mini_cluster.cc:1428] Setting key 2cb32bb7096fb4c4cd5eab2fcce9e5e4
I20260812 21:03:54.370059 21999 raft_consensus.cc:504] T 00000000000000000000000000000000 P f3e09e0d3a6842a7a2994f8df349ea12 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3e09e0d3a6842a7a2994f8df349ea12" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 39283 } }
I20260812 21:03:54.370241 21999 leader_election.cc:302] T 00000000000000000000000000000000 P f3e09e0d3a6842a7a2994f8df349ea12 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f3e09e0d3a6842a7a2994f8df349ea12; no voters:
I20260812 21:03:54.370395 21999 leader_election.cc:288] T 00000000000000000000000000000000 P f3e09e0d3a6842a7a2994f8df349ea12 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 21:03:54.370427 22006 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f3e09e0d3a6842a7a2994f8df349ea12 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:03:54.370560 22006 raft_consensus.cc:686] T 00000000000000000000000000000000 P f3e09e0d3a6842a7a2994f8df349ea12 [term 1 LEADER]: Becoming Leader. State: Replica: f3e09e0d3a6842a7a2994f8df349ea12, State: Running, Role: LEADER
I20260812 21:03:54.370668 21999 sys_catalog.cc:564] T 00000000000000000000000000000000 P f3e09e0d3a6842a7a2994f8df349ea12 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 21:03:54.370688 22006 consensus_queue.cc:237] T 00000000000000000000000000000000 P f3e09e0d3a6842a7a2994f8df349ea12 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3e09e0d3a6842a7a2994f8df349ea12" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 39283 } }
I20260812 21:03:54.371275 22010 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3e09e0d3a6842a7a2994f8df349ea12 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f3e09e0d3a6842a7a2994f8df349ea12. Latest consensus state: current_term: 1 leader_uuid: "f3e09e0d3a6842a7a2994f8df349ea12" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3e09e0d3a6842a7a2994f8df349ea12" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 39283 } } }
I20260812 21:03:54.371359 22010 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3e09e0d3a6842a7a2994f8df349ea12 [sys.catalog]: This master's current role is: LEADER
I20260812 21:03:54.371737 22022 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 21:03:54.371745 22009 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3e09e0d3a6842a7a2994f8df349ea12 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f3e09e0d3a6842a7a2994f8df349ea12" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3e09e0d3a6842a7a2994f8df349ea12" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 39283 } } }
I20260812 21:03:54.372176 22009 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3e09e0d3a6842a7a2994f8df349ea12 [sys.catalog]: This master's current role is: LEADER
I20260812 21:03:54.372344 22022 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 21:03:54.373970 22022 catalog_manager.cc:1347] Generated new cluster ID: 928b8279fbc7464da33c8ebd935f0afa
I20260812 21:03:54.374024 22022 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 21:03:54.375428 22004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:54.370000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46094 (local address 127.11.254.190:39283)
0812 21:03:54.370202 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0812 21:03:54.370210 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 21:03:54.370235 (+ 25us) server_negotiation.cc:407] Connection header received
0812 21:03:54.370341 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:03:54.370348 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:03:54.370420 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:03:54.370616 (+ 196us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:03:54.370982 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:54.372502 (+ 1520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:54.373445 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:54.374036 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:54.374155 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:54.374812 (+ 657us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:03:54.374816 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:03:54.374817 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 21:03:54.374996 (+ 179us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:03:54.375034 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 21:03:54.375045 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:03:54.375100 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0812 21:03:54.375179 (+ 79us) server_negotiation.cc:299] Negotiation successful
0812 21:03:54.375259 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":86,"threads_started":1}
I20260812 21:03:54.376245 12282 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 21:03:54.386814 22022 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 21:03:54.387485 22022 catalog_manager.cc:1504] Loading token signing keys...
I20260812 21:03:54.396507 22022 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f3e09e0d3a6842a7a2994f8df349ea12: Generated new TSK 0
I20260812 21:03:54.396735 22022 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 21:03:54.411443 22004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:54.407620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46098 (local address 127.11.254.190:39283)
0812 21:03:54.407675 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0812 21:03:54.407677 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:03:54.407740 (+ 63us) server_negotiation.cc:407] Connection header received
0812 21:03:54.407833 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:03:54.407834 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:03:54.407845 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:03:54.407906 (+ 61us) server_negotiation.cc:226] Negotiated authn=JWT
0812 21:03:54.408046 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:54.408626 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:54.409747 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:54.409974 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:54.410023 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:54.410216 (+ 193us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0812 21:03:54.411222 (+ 1006us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34641/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0812 21:03:54.411363 (+ 141us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:46098: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34641/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":29}
W20260812 21:03:54.411526 22004 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:46098: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34641/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260812 21:03:54.411546 22049 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:03:54.407532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:39283 (local address 127.0.0.1:46098)
0812 21:03:54.407680 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0812 21:03:54.407691 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 21:03:54.407761 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:03:54.407915 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:03:54.407916 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:03:54.407917 (+ 1us) client_negotiation.cc:189] Negotiated authn=JWT
0812 21:03:54.407990 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:54.407993 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:54.408704 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:54.408705 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:54.409657 (+ 952us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:54.409661 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:54.410043 (+ 382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:54.410044 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:54.410117 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:54.410124 (+ 7us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0812 21:03:54.411319 (+ 1195us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34641/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0812 21:03:54.411437 (+ 118us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.11.254.190:39283: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34641/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":83,"threads_started":1}
I20260812 21:03:54.412130 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 21875
E20260812 21:03:54.434672 22050 webserver.cc:533] Webserver: sslize failed:
E20260812 21:03:54.434875 22050 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-12T21:03:54Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (259 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-12T21:03:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T21:03:54Z Disabled control of system clock
I20260812 21:03:54.483394 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786568607354921-12282-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786568607354921-12282-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786568607354921-12282-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.254.190:43803
--webserver_interface=127.11.254.190
--webserver_port=0
--builtin_ntp_servers=127.11.254.148:33495
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.254.190:43803
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786568607354921-12282-0/testchainca.cert with env {}
W20260812 21:03:54.575424 22060 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:03:54.575636 22060 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:03:54.575657 22060 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:03:54.576790 22060 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 21:03:54.576833 22060 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:03:54.576843 22060 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:03:54.576865 22060 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 21:03:54.576889 22060 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260812 21:03:54.576911 22060 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260812 21:03:54.576926 22060 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260812 21:03:54.578500 22060 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.254.148:33495
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786568607354921-12282-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786568607354921-12282-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.254.190:43803
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.254.190:43803
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.254.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786568607354921-12282-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22060
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786568607354921-12282-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:03:54.578737 22060 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:03:54.578939 22060 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:03:54.583905 22068 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:03:54.583887 22069 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:03:54.583884 22074 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:03:54.584124 22060 server_base.cc:1034] running on GCE node
I20260812 21:03:54.584407 22060 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:03:54.584901 22060 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:03:54.586117 22060 hybrid_clock.cc:648] HybridClock initialized: now 1786568634586085 us; error 55 us; skew 500 ppm
I20260812 21:03:54.587631 22060 webserver.cc:460] Webserver started at http://127.11.254.190:42089/ using document root <none> and password file <none>
I20260812 21:03:54.587880 22060 fs_manager.cc:359] Metadata directory not provided
I20260812 21:03:54.587935 22060 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:03:54.588080 22060 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:03:54.589219 22060 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786568607354921-12282-0/minicluster-data/master-0/data/instance:
uuid: "2ce8b8dc4ab647d8b9353df433ee6e02"
format_stamp: "Formatted at 2026-08-12 21:03:54 on dist-test-slave-vpvm"
server_key: "557a6fd9d0a6300924a54a45e6a8409d"
server_key_iv: "f5dd2b641291a94f548ab39a10993fc6"
server_key_version: "encryptionkey@0"
I20260812 21:03:54.589576 22060 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786568607354921-12282-0/minicluster-data/master-0/wal/instance:
uuid: "2ce8b8dc4ab647d8b9353df433ee6e02"
format_stamp: "Formatted at 2026-08-12 21:03:54 on dist-test-slave-vpvm"
server_key: "557a6fd9d0a6300924a54a45e6a8409d"
server_key_iv: "f5dd2b641291a94f548ab39a10993fc6"
server_key_version: "encryptionkey@0"
I20260812 21:03:54.591023 22060 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 21:03:54.591776 22085 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:03:54.591917 22060 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 21:03:54.592006 22060 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786568607354921-12282-0/minicluster-data/master-0/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786568607354921-12282-0/minicluster-data/master-0/wal
uuid: "2ce8b8dc4ab647d8b9353df433ee6e02"
format_stamp: "Formatted at 2026-08-12 21:03:54 on dist-test-slave-vpvm"
server_key: "557a6fd9d0a6300924a54a45e6a8409d"
server_key_iv: "f5dd2b641291a94f548ab39a10993fc6"
server_key_version: "encryptionkey@0"
I20260812 21:03:54.592078 22060 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786568607354921-12282-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786568607354921-12282-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786568607354921-12282-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:03:54.630810 22060 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:03:54.631175 22060 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:03:54.631313 22060 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:03:54.635180 22060 rpc_server.cc:307] RPC server started. Bound to: 127.11.254.190:43803
I20260812 21:03:54.635237 22167 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.254.190:43803 every 8 connection(s)
I20260812 21:03:54.635563 22060 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
I20260812 21:03:54.636616 22168 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:03:54.639002 22168 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ce8b8dc4ab647d8b9353df433ee6e02: Bootstrap starting.
I20260812 21:03:54.639597 22168 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2ce8b8dc4ab647d8b9353df433ee6e02: Neither blocks nor log segments found. Creating new log.
I20260812 21:03:54.639942 22168 log.cc:824] T 00000000000000000000000000000000 P 2ce8b8dc4ab647d8b9353df433ee6e02: Log is configured to *not* fsync() on all Append() calls
I20260812 21:03:54.640771 22168 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ce8b8dc4ab647d8b9353df433ee6e02: No bootstrap required, opened a new log
I20260812 21:03:54.641258 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 22060
I20260812 21:03:54.641359 12282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786568607354921-12282-0/minicluster-data/master-0/wal/instance
I20260812 21:03:54.641528 12282 external_mini_cluster.cc:1428] Setting key 7f5045f3fa8c1a230e8f606fcc826ab7
I20260812 21:03:54.642946 22168 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2ce8b8dc4ab647d8b9353df433ee6e02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ce8b8dc4ab647d8b9353df433ee6e02" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 43803 } }
I20260812 21:03:54.643142 22168 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2ce8b8dc4ab647d8b9353df433ee6e02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:03:54.643244 22168 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2ce8b8dc4ab647d8b9353df433ee6e02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ce8b8dc4ab647d8b9353df433ee6e02, State: Initialized, Role: FOLLOWER
I20260812 21:03:54.643383 22168 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2ce8b8dc4ab647d8b9353df433ee6e02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ce8b8dc4ab647d8b9353df433ee6e02" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 43803 } }
I20260812 21:03:54.643471 22168 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2ce8b8dc4ab647d8b9353df433ee6e02 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 21:03:54.643510 22168 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2ce8b8dc4ab647d8b9353df433ee6e02 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 21:03:54.643556 22168 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2ce8b8dc4ab647d8b9353df433ee6e02 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:03:54.644300 22168 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2ce8b8dc4ab647d8b9353df433ee6e02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ce8b8dc4ab647d8b9353df433ee6e02" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 43803 } }
I20260812 21:03:54.644411 22168 leader_election.cc:302] T 00000000000000000000000000000000 P 2ce8b8dc4ab647d8b9353df433ee6e02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2ce8b8dc4ab647d8b9353df433ee6e02; no voters:
I20260812 21:03:54.644565 22168 leader_election.cc:288] T 00000000000000000000000000000000 P 2ce8b8dc4ab647d8b9353df433ee6e02 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 21:03:54.644599 22178 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2ce8b8dc4ab647d8b9353df433ee6e02 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:03:54.644757 22178 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2ce8b8dc4ab647d8b9353df433ee6e02 [term 1 LEADER]: Becoming Leader. State: Replica: 2ce8b8dc4ab647d8b9353df433ee6e02, State: Running, Role: LEADER
I20260812 21:03:54.644815 22168 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2ce8b8dc4ab647d8b9353df433ee6e02 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 21:03:54.644891 22178 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2ce8b8dc4ab647d8b9353df433ee6e02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ce8b8dc4ab647d8b9353df433ee6e02" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 43803 } }
I20260812 21:03:54.645223 22178 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ce8b8dc4ab647d8b9353df433ee6e02 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2ce8b8dc4ab647d8b9353df433ee6e02. Latest consensus state: current_term: 1 leader_uuid: "2ce8b8dc4ab647d8b9353df433ee6e02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ce8b8dc4ab647d8b9353df433ee6e02" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 43803 } } }
I20260812 21:03:54.645296 22180 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ce8b8dc4ab647d8b9353df433ee6e02 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2ce8b8dc4ab647d8b9353df433ee6e02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ce8b8dc4ab647d8b9353df433ee6e02" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 43803 } } }
I20260812 21:03:54.645349 22180 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ce8b8dc4ab647d8b9353df433ee6e02 [sys.catalog]: This master's current role is: LEADER
I20260812 21:03:54.645346 22178 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ce8b8dc4ab647d8b9353df433ee6e02 [sys.catalog]: This master's current role is: LEADER
I20260812 21:03:54.645615 22190 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 21:03:54.646220 22190 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 21:03:54.647708 22190 catalog_manager.cc:1347] Generated new cluster ID: 667e29f84bff4838bf477953809c70c5
I20260812 21:03:54.647763 22190 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 21:03:54.648307 22177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:54.642022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52514 (local address 127.11.254.190:43803)
0812 21:03:54.642274 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0812 21:03:54.642282 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 21:03:54.642316 (+ 34us) server_negotiation.cc:407] Connection header received
0812 21:03:54.642910 (+ 594us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:03:54.642919 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:03:54.643001 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:03:54.643182 (+ 181us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:03:54.643396 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:54.645110 (+ 1714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:54.646288 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:54.646773 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:54.646884 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:54.647600 (+ 716us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:03:54.647603 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:03:54.647604 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 21:03:54.647807 (+ 203us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:03:54.647851 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 21:03:54.647866 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:03:54.647927 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0812 21:03:54.648017 (+ 90us) server_negotiation.cc:299] Negotiation successful
0812 21:03:54.648097 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":114,"threads_started":1}
I20260812 21:03:54.649134 12282 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 21:03:54.674769 22190 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 21:03:54.675312 22190 catalog_manager.cc:1504] Loading token signing keys...
I20260812 21:03:54.680058 22190 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2ce8b8dc4ab647d8b9353df433ee6e02: Generated new TSK 0
I20260812 21:03:54.680269 22190 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 21:03:54.746603 22177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:54.691088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52522 (local address 127.11.254.190:43803)
0812 21:03:54.691148 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0812 21:03:54.691150 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:03:54.691203 (+ 53us) server_negotiation.cc:407] Connection header received
0812 21:03:54.691282 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:03:54.691283 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:03:54.691293 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:03:54.691353 (+ 60us) server_negotiation.cc:226] Negotiated authn=JWT
0812 21:03:54.691484 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:54.691965 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:54.693095 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:54.693283 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:54.693334 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:54.693505 (+ 171us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0812 21:03:54.746248 (+ 52743us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0812 21:03:54.746518 (+ 270us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:52522: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":37}
W20260812 21:03:54.746691 22177 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:52522: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260812 21:03:54.746701 22225 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:03:54.690968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:43803 (local address 127.0.0.1:52522)
0812 21:03:54.691135 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0812 21:03:54.691147 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 21:03:54.691218 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:03:54.691367 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:03:54.691367 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:03:54.691369 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0812 21:03:54.691431 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:54.691434 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:54.692047 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:54.692048 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:54.693008 (+ 960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:54.693011 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:54.693354 (+ 343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:54.693355 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:54.693416 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:54.693422 (+ 6us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0812 21:03:54.746402 (+ 52980us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0812 21:03:54.746604 (+ 202us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.11.254.190:43803: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":93,"threads_started":1}
I20260812 21:03:54.746991 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 22060
2026-08-12T21:03:54Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (295 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (92 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (91 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568607354921-12282-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 21:03:54 dist-test-slave-vpvm krb5kdc[22245](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 21:03:54 dist-test-slave-vpvm krb5kdc[22245](info): set up 2 sockets
Aug 12 21:03:54 dist-test-slave-vpvm krb5kdc[22245](info): commencing operation
krb5kdc: starting...
W20260812 21:03:56.171407 12282 mini_kdc.cc:121] Time spent starting KDC: real 1.217s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568607354921-12282-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568607354921-12282-0/krb5kdc/test-user.keytab.
Aug 12 21:03:56 dist-test-slave-vpvm krb5kdc[22245](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568636, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T21:03:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T21:03:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.254.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.254.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.254.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.190.keytab.
Entry for principal kudu/127.11.254.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.190.keytab.
I20260812 21:03:56.299042 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568607354921-12282-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568607354921-12282-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568607354921-12282-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.254.190:38587
--webserver_interface=127.11.254.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.190.keytab
--principal=kudu/127.11.254.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.254.148:40169
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.254.190:38587
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568607354921-12282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568607354921-12282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568607354921-12282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:03:56.382206 22271 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:03:56.382433 22271 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:03:56.382457 22271 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:03:56.383641 22271 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 21:03:56.383683 22271 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:03:56.383697 22271 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:03:56.383708 22271 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 21:03:56.383718 22271 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 21:03:56.385200 22271 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.254.148:40169
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568607354921-12282-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568607354921-12282-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.254.190:38587
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.190.keytab
--principal=kudu/127.11.254.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.254.190:38587
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.254.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22271
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568607354921-12282-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:03:56.385427 22271 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:03:56.385627 22271 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:03:56.390745 22279 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:03:56.390739 22280 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:03:56.390739 22285 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:03:56.390826 22271 server_base.cc:1034] running on GCE node
I20260812 21:03:56.391266 22271 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:03:56.391600 22271 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:03:56.392803 22271 hybrid_clock.cc:648] HybridClock initialized: now 1786568636392783 us; error 33 us; skew 500 ppm
Aug 12 21:03:56 dist-test-slave-vpvm krb5kdc[22245](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568636, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:03:56.399384 22271 init.cc:377] Logged in from keytab as kudu/127.11.254.190@KRBTEST.COM (short username kudu)
I20260812 21:03:56.400033 22271 webserver.cc:460] Webserver started at http://127.11.254.190:35487/ using document root <none> and password file <none>
I20260812 21:03:56.400331 22271 fs_manager.cc:359] Metadata directory not provided
I20260812 21:03:56.400463 22271 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:03:56.400672 22271 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:03:56.402288 22271 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568607354921-12282-0/minicluster-data/master-0/data/instance:
uuid: "0f2541ebe5384a9eb0f45cd491a3d245"
format_stamp: "Formatted at 2026-08-12 21:03:56 on dist-test-slave-vpvm"
server_key: "8d756a91c4eee92f07c1e1c0f9f64d0a"
server_key_iv: "bfab052c2617b95583f7a7ed8b77d3a5"
server_key_version: "encryptionkey@0"
I20260812 21:03:56.402767 22271 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568607354921-12282-0/minicluster-data/master-0/wal/instance:
uuid: "0f2541ebe5384a9eb0f45cd491a3d245"
format_stamp: "Formatted at 2026-08-12 21:03:56 on dist-test-slave-vpvm"
server_key: "8d756a91c4eee92f07c1e1c0f9f64d0a"
server_key_iv: "bfab052c2617b95583f7a7ed8b77d3a5"
server_key_version: "encryptionkey@0"
I20260812 21:03:56.406298 22271 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 21:03:56.408219 22301 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:03:56.408411 22271 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 21:03:56.408509 22271 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568607354921-12282-0/minicluster-data/master-0/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568607354921-12282-0/minicluster-data/master-0/wal
uuid: "0f2541ebe5384a9eb0f45cd491a3d245"
format_stamp: "Formatted at 2026-08-12 21:03:56 on dist-test-slave-vpvm"
server_key: "8d756a91c4eee92f07c1e1c0f9f64d0a"
server_key_iv: "bfab052c2617b95583f7a7ed8b77d3a5"
server_key_version: "encryptionkey@0"
I20260812 21:03:56.408592 22271 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568607354921-12282-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568607354921-12282-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568607354921-12282-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:03:56.419445 22271 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:03:56.421264 22271 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:03:56.421386 22271 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:03:56.425343 22271 rpc_server.cc:307] RPC server started. Bound to: 127.11.254.190:38587
I20260812 21:03:56.425388 22385 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.254.190:38587 every 8 connection(s)
I20260812 21:03:56.425742 22271 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
I20260812 21:03:56.426434 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 22271
I20260812 21:03:56.426548 12282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568607354921-12282-0/minicluster-data/master-0/wal/instance
I20260812 21:03:56.426756 12282 external_mini_cluster.cc:1428] Setting key a75f40bbeec4c3052debcbead3dc6720
I20260812 21:03:56.426841 22386 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:03:56.430083 22386 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0f2541ebe5384a9eb0f45cd491a3d245: Bootstrap starting.
I20260812 21:03:56.430814 22386 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0f2541ebe5384a9eb0f45cd491a3d245: Neither blocks nor log segments found. Creating new log.
I20260812 21:03:56.431098 22386 log.cc:824] T 00000000000000000000000000000000 P 0f2541ebe5384a9eb0f45cd491a3d245: Log is configured to *not* fsync() on all Append() calls
I20260812 21:03:56.431816 22386 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0f2541ebe5384a9eb0f45cd491a3d245: No bootstrap required, opened a new log
Aug 12 21:03:56 dist-test-slave-vpvm krb5kdc[22245](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568636, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.254.190@KRBTEST.COM
I20260812 21:03:56.433810 22386 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0f2541ebe5384a9eb0f45cd491a3d245 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f2541ebe5384a9eb0f45cd491a3d245" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 38587 } }
I20260812 21:03:56.433981 22386 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0f2541ebe5384a9eb0f45cd491a3d245 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:03:56.434031 22386 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0f2541ebe5384a9eb0f45cd491a3d245 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f2541ebe5384a9eb0f45cd491a3d245, State: Initialized, Role: FOLLOWER
I20260812 21:03:56.434162 22386 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0f2541ebe5384a9eb0f45cd491a3d245 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f2541ebe5384a9eb0f45cd491a3d245" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 38587 } }
I20260812 21:03:56.434254 22386 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0f2541ebe5384a9eb0f45cd491a3d245 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 21:03:56.434298 22386 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0f2541ebe5384a9eb0f45cd491a3d245 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 21:03:56.434348 22386 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0f2541ebe5384a9eb0f45cd491a3d245 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:03:56.434988 22386 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0f2541ebe5384a9eb0f45cd491a3d245 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f2541ebe5384a9eb0f45cd491a3d245" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 38587 } }
I20260812 21:03:56.435082 22386 leader_election.cc:302] T 00000000000000000000000000000000 P 0f2541ebe5384a9eb0f45cd491a3d245 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0f2541ebe5384a9eb0f45cd491a3d245; no voters:
I20260812 21:03:56.435248 22386 leader_election.cc:288] T 00000000000000000000000000000000 P 0f2541ebe5384a9eb0f45cd491a3d245 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 21:03:56.435293 22393 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0f2541ebe5384a9eb0f45cd491a3d245 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:03:56.435457 22393 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0f2541ebe5384a9eb0f45cd491a3d245 [term 1 LEADER]: Becoming Leader. State: Replica: 0f2541ebe5384a9eb0f45cd491a3d245, State: Running, Role: LEADER
I20260812 21:03:56.435540 22386 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0f2541ebe5384a9eb0f45cd491a3d245 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 21:03:56.435593 22393 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0f2541ebe5384a9eb0f45cd491a3d245 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f2541ebe5384a9eb0f45cd491a3d245" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 38587 } }
I20260812 21:03:56.435985 22393 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0f2541ebe5384a9eb0f45cd491a3d245 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0f2541ebe5384a9eb0f45cd491a3d245. Latest consensus state: current_term: 1 leader_uuid: "0f2541ebe5384a9eb0f45cd491a3d245" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f2541ebe5384a9eb0f45cd491a3d245" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 38587 } } }
I20260812 21:03:56.436002 22400 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0f2541ebe5384a9eb0f45cd491a3d245 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0f2541ebe5384a9eb0f45cd491a3d245" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f2541ebe5384a9eb0f45cd491a3d245" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 38587 } } }
I20260812 21:03:56.436151 22393 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0f2541ebe5384a9eb0f45cd491a3d245 [sys.catalog]: This master's current role is: LEADER
I20260812 21:03:56.436414 22400 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0f2541ebe5384a9eb0f45cd491a3d245 [sys.catalog]: This master's current role is: LEADER
I20260812 21:03:56.437326 22389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:56.427300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59144 (local address 127.11.254.190:38587)
0812 21:03:56.427477 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0812 21:03:56.427485 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 21:03:56.427511 (+ 26us) server_negotiation.cc:407] Connection header received
0812 21:03:56.427939 (+ 428us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:03:56.427947 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:03:56.427996 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:03:56.428112 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:03:56.428881 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:56.429765 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:56.430775 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:56.431266 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:56.431370 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:56.433712 (+ 2342us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:03:56.433716 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:03:56.433717 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:03:56.433729 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:03:56.435484 (+ 1755us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:56.435926 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:56.435929 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:56.435929 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:56.436042 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:56.436272 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:56.436273 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:56.436273 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:56.436632 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:03:56.436798 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0812 21:03:56.436935 (+ 137us) server_negotiation.cc:299] Negotiation successful
0812 21:03:56.437079 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":75,"threads_started":1}
W20260812 21:03:56.437472 22425 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0f2541ebe5384a9eb0f45cd491a3d245: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 21:03:56.437541 22425 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 21:03:56.437600 22413 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 21:03:56.437753 22413 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 21:03:56.439127 22413 catalog_manager.cc:1347] Generated new cluster ID: f05fb6a4bb774d3f8282ba3e9b57a05d
I20260812 21:03:56.439175 22413 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 21:03:56.445917 22413 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 21:03:56.446564 22413 catalog_manager.cc:1504] Loading token signing keys...
I20260812 21:03:56.453652 22413 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0f2541ebe5384a9eb0f45cd491a3d245: Generated new TSK 0
I20260812 21:03:56.453892 22413 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.254.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.254.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.254.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.129.keytab.
Entry for principal kudu/127.11.254.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.129.keytab.
I20260812 21:03:56.484481 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568607354921-12282-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568607354921-12282-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568607354921-12282-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568607354921-12282-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.254.129:0
--local_ip_for_outbound_sockets=127.11.254.129
--webserver_interface=127.11.254.129
--webserver_port=0
--tserver_master_addrs=127.11.254.190:38587
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.129.keytab
--principal=kudu/127.11.254.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.254.148:40169
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568607354921-12282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568607354921-12282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568607354921-12282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:03:56.567843 22435 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:03:56.568061 22435 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:03:56.568092 22435 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:03:56.569249 22435 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:03:56.569290 22435 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:03:56.569315 22435 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.254.129
I20260812 21:03:56.570883 22435 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.254.148:40169
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568607354921-12282-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568607354921-12282-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.129.keytab
--principal=kudu/127.11.254.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.254.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568607354921-12282-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.254.129
--webserver_port=0
--tserver_master_addrs=127.11.254.190:38587
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22435
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.254.129
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568607354921-12282-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:03:56.571135 22435 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:03:56.571376 22435 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:03:56.576303 22449 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:03:56.576311 22445 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:03:56.576586 22444 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:03:56.576555 22435 server_base.cc:1034] running on GCE node
I20260812 21:03:56.576792 22435 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:03:56.577015 22435 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:03:56.578150 22435 hybrid_clock.cc:648] HybridClock initialized: now 1786568636578138 us; error 29 us; skew 500 ppm
Aug 12 21:03:56 dist-test-slave-vpvm krb5kdc[22245](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568636, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:03:56.580655 22435 init.cc:377] Logged in from keytab as kudu/127.11.254.129@KRBTEST.COM (short username kudu)
I20260812 21:03:56.581070 22435 webserver.cc:460] Webserver started at http://127.11.254.129:34099/ using document root <none> and password file <none>
I20260812 21:03:56.581275 22435 fs_manager.cc:359] Metadata directory not provided
I20260812 21:03:56.581321 22435 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:03:56.581429 22435 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:03:56.582508 22435 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568607354921-12282-0/minicluster-data/ts-0/data/instance:
uuid: "1af1569baff24a99864359432fa6af82"
format_stamp: "Formatted at 2026-08-12 21:03:56 on dist-test-slave-vpvm"
server_key: "6829e3a8356f1e3b225437fb798bb40b"
server_key_iv: "a44f0fc7a32b5230e953fef871682ae2"
server_key_version: "encryptionkey@0"
I20260812 21:03:56.582857 22435 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568607354921-12282-0/minicluster-data/ts-0/wal/instance:
uuid: "1af1569baff24a99864359432fa6af82"
format_stamp: "Formatted at 2026-08-12 21:03:56 on dist-test-slave-vpvm"
server_key: "6829e3a8356f1e3b225437fb798bb40b"
server_key_iv: "a44f0fc7a32b5230e953fef871682ae2"
server_key_version: "encryptionkey@0"
I20260812 21:03:56.584172 22435 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 21:03:56.584942 22458 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:03:56.585127 22435 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 21:03:56.585218 22435 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568607354921-12282-0/minicluster-data/ts-0/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568607354921-12282-0/minicluster-data/ts-0/wal
uuid: "1af1569baff24a99864359432fa6af82"
format_stamp: "Formatted at 2026-08-12 21:03:56 on dist-test-slave-vpvm"
server_key: "6829e3a8356f1e3b225437fb798bb40b"
server_key_iv: "a44f0fc7a32b5230e953fef871682ae2"
server_key_version: "encryptionkey@0"
I20260812 21:03:56.585281 22435 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568607354921-12282-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568607354921-12282-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568607354921-12282-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:03:56.594244 22435 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:03:56.596129 22435 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:03:56.596268 22435 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:03:56.596478 22435 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:03:56.596822 22435 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:03:56.596858 22435 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:03:56.596887 22435 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:03:56.596910 22435 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:03:56.602607 22435 rpc_server.cc:307] RPC server started. Bound to: 127.11.254.129:34329
I20260812 21:03:56.602648 22660 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.254.129:34329 every 8 connection(s)
I20260812 21:03:56.602990 22435 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568607354921-12282-0/minicluster-data/ts-0/data/info.pb
Aug 12 21:03:56 dist-test-slave-vpvm krb5kdc[22245](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568636, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.129@KRBTEST.COM for kudu/127.11.254.190@KRBTEST.COM
I20260812 21:03:56.610272 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 22435
I20260812 21:03:56.610381 12282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568607354921-12282-0/minicluster-data/ts-0/wal/instance
I20260812 21:03:56.610591 12282 external_mini_cluster.cc:1428] Setting key 4203c9821f453411087e1dd153a19e21
I20260812 21:03:56.612556 22389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:56.603732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.129:38383 (local address 127.11.254.190:38587)
0812 21:03:56.603791 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0812 21:03:56.603793 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:03:56.603976 (+ 183us) server_negotiation.cc:407] Connection header received
0812 21:03:56.604982 (+ 1006us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:03:56.604984 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:03:56.604999 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:03:56.605053 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:03:56.605705 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:56.606306 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:56.607308 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:56.607521 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:56.607578 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:56.609694 (+ 2116us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:03:56.609695 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:03:56.609696 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:03:56.609704 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:03:56.611126 (+ 1422us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:56.611550 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:56.611551 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:56.611552 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:56.611622 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:56.611904 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:56.611905 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:56.611905 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:56.612043 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:03:56.612103 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0812 21:03:56.612363 (+ 260us) server_negotiation.cc:299] Negotiation successful
0812 21:03:56.612463 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260812 21:03:56.612756 22668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:56.603711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:38587 (local address 127.11.254.129:38383)
0812 21:03:56.603863 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0812 21:03:56.603899 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0812 21:03:56.604865 (+ 966us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:03:56.605127 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:03:56.605130 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:03:56.605402 (+ 272us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:03:56.605613 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:56.605618 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:56.606396 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:56.606397 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:56.607218 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:56.607220 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:56.607602 (+ 382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:56.607603 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:56.607720 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:56.608194 (+ 474us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:03:56.608202 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:03:56.609598 (+ 1396us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:03:56.611196 (+ 1598us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:03:56.611206 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:03:56.611208 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:03:56.611478 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:03:56.611684 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:03:56.611684 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:03:56.611684 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:03:56.611809 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:03:56.612102 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:03:56.612106 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:03:56.612233 (+ 127us) client_negotiation.cc:769] Sending connection context
0812 21:03:56.612320 (+ 87us) client_negotiation.cc:240] Negotiation successful
0812 21:03:56.612486 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":81,"threads_started":1}
I20260812 21:03:56.613250 22664 heartbeater.cc:347] Connected to a master server at 127.11.254.190:38587
I20260812 21:03:56.613355 22664 heartbeater.cc:464] Registering TS with master...
I20260812 21:03:56.613561 22664 heartbeater.cc:511] Master 127.11.254.190:38587 requested a full tablet report, sending...
I20260812 21:03:56.614135 22328 ts_manager.cc:194] Registered new tserver with Master: 1af1569baff24a99864359432fa6af82 (127.11.254.129:34329)
I20260812 21:03:56.614730 22328 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.254.129@KRBTEST.COM'} at 127.11.254.129:38383
WARNING: no policy specified for kudu/127.11.254.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.254.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.254.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.130.keytab.
Entry for principal kudu/127.11.254.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.130.keytab.
I20260812 21:03:56.636771 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568607354921-12282-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568607354921-12282-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568607354921-12282-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568607354921-12282-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.254.130:0
--local_ip_for_outbound_sockets=127.11.254.130
--webserver_interface=127.11.254.130
--webserver_port=0
--tserver_master_addrs=127.11.254.190:38587
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.130.keytab
--principal=kudu/127.11.254.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.254.148:40169
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568607354921-12282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568607354921-12282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568607354921-12282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:03:56.722003 22672 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:03:56.722236 22672 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:03:56.722270 22672 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:03:56.723572 22672 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:03:56.723626 22672 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:03:56.723664 22672 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.254.130
I20260812 21:03:56.725260 22672 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.254.148:40169
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568607354921-12282-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568607354921-12282-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.130.keytab
--principal=kudu/127.11.254.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.254.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568607354921-12282-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.254.130
--webserver_port=0
--tserver_master_addrs=127.11.254.190:38587
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22672
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.254.130
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568607354921-12282-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:03:56.725533 22672 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:03:56.725777 22672 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:03:56.731174 22681 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:03:56.731134 22683 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:03:56.731204 22672 server_base.cc:1034] running on GCE node
W20260812 21:03:56.731149 22679 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:03:56.731644 22672 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:03:56.731904 22672 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:03:56.733067 22672 hybrid_clock.cc:648] HybridClock initialized: now 1786568636733051 us; error 51 us; skew 500 ppm
Aug 12 21:03:56 dist-test-slave-vpvm krb5kdc[22245](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568636, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:03:56.735873 22672 init.cc:377] Logged in from keytab as kudu/127.11.254.130@KRBTEST.COM (short username kudu)
I20260812 21:03:56.736353 22672 webserver.cc:460] Webserver started at http://127.11.254.130:45205/ using document root <none> and password file <none>
I20260812 21:03:56.736631 22672 fs_manager.cc:359] Metadata directory not provided
I20260812 21:03:56.736685 22672 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:03:56.736836 22672 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:03:56.737977 22672 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568607354921-12282-0/minicluster-data/ts-1/data/instance:
uuid: "3aef430cded14d709c28c66109df2f75"
format_stamp: "Formatted at 2026-08-12 21:03:56 on dist-test-slave-vpvm"
server_key: "aaadba9681fb1fd816b75d57871f98dc"
server_key_iv: "c7e21231e313e310d571992715ca0f33"
server_key_version: "encryptionkey@0"
I20260812 21:03:56.738341 22672 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568607354921-12282-0/minicluster-data/ts-1/wal/instance:
uuid: "3aef430cded14d709c28c66109df2f75"
format_stamp: "Formatted at 2026-08-12 21:03:56 on dist-test-slave-vpvm"
server_key: "aaadba9681fb1fd816b75d57871f98dc"
server_key_iv: "c7e21231e313e310d571992715ca0f33"
server_key_version: "encryptionkey@0"
I20260812 21:03:56.739717 22672 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 21:03:56.740458 22696 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:03:56.740649 22672 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 21:03:56.740737 22672 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568607354921-12282-0/minicluster-data/ts-1/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568607354921-12282-0/minicluster-data/ts-1/wal
uuid: "3aef430cded14d709c28c66109df2f75"
format_stamp: "Formatted at 2026-08-12 21:03:56 on dist-test-slave-vpvm"
server_key: "aaadba9681fb1fd816b75d57871f98dc"
server_key_iv: "c7e21231e313e310d571992715ca0f33"
server_key_version: "encryptionkey@0"
I20260812 21:03:56.740808 22672 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568607354921-12282-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568607354921-12282-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568607354921-12282-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:03:56.748526 22672 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:03:56.750622 22672 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:03:56.750768 22672 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:03:56.750985 22672 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:03:56.751286 22672 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:03:56.751320 22672 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:03:56.751349 22672 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:03:56.751372 22672 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:03:56.756871 22672 rpc_server.cc:307] RPC server started. Bound to: 127.11.254.130:42807
I20260812 21:03:56.756946 22882 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.254.130:42807 every 8 connection(s)
I20260812 21:03:56.757239 22672 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568607354921-12282-0/minicluster-data/ts-1/data/info.pb
I20260812 21:03:56.762701 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 22672
I20260812 21:03:56.762809 12282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568607354921-12282-0/minicluster-data/ts-1/wal/instance
I20260812 21:03:56.763000 12282 external_mini_cluster.cc:1428] Setting key 808790bcabd135f23c9d777dad35b2f6
Aug 12 21:03:56 dist-test-slave-vpvm krb5kdc[22245](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568636, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.130@KRBTEST.COM for kudu/127.11.254.190@KRBTEST.COM
I20260812 21:03:56.766548 22389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:56.758060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.130:53097 (local address 127.11.254.190:38587)
0812 21:03:56.758135 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0812 21:03:56.758137 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:03:56.758298 (+ 161us) server_negotiation.cc:407] Connection header received
0812 21:03:56.759228 (+ 930us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:03:56.759230 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:03:56.759240 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:03:56.759291 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:03:56.759881 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:56.760388 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:56.761398 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:56.761597 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:56.761646 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:56.763932 (+ 2286us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:03:56.763934 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:03:56.763934 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:03:56.763942 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:03:56.765150 (+ 1208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:56.765564 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:56.765565 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:56.765566 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:56.765624 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:56.765854 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:56.765855 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:56.765855 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:56.766012 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:03:56.766079 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 21:03:56.766348 (+ 269us) server_negotiation.cc:299] Negotiation successful
0812 21:03:56.766445 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260812 21:03:56.766712 22886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:56.758002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:38587 (local address 127.11.254.130:53097)
0812 21:03:56.758178 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0812 21:03:56.758218 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 21:03:56.759141 (+ 923us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:03:56.759348 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:03:56.759352 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:03:56.759621 (+ 269us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:03:56.759820 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:56.759824 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:56.760478 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:56.760479 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:56.761328 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:56.761331 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:56.761658 (+ 327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:56.761659 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:56.761761 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:56.762255 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:03:56.762264 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:03:56.763850 (+ 1586us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:03:56.765227 (+ 1377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:03:56.765235 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:03:56.765238 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:03:56.765482 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:03:56.765683 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:03:56.765684 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:03:56.765684 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:03:56.765794 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:03:56.766066 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:03:56.766072 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:03:56.766224 (+ 152us) client_negotiation.cc:769] Sending connection context
0812 21:03:56.766316 (+ 92us) client_negotiation.cc:240] Negotiation successful
0812 21:03:56.766487 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":80,"threads_started":1}
I20260812 21:03:56.767241 22883 heartbeater.cc:347] Connected to a master server at 127.11.254.190:38587
I20260812 21:03:56.767349 22883 heartbeater.cc:464] Registering TS with master...
I20260812 21:03:56.767544 22883 heartbeater.cc:511] Master 127.11.254.190:38587 requested a full tablet report, sending...
I20260812 21:03:56.767954 22328 ts_manager.cc:194] Registered new tserver with Master: 3aef430cded14d709c28c66109df2f75 (127.11.254.130:42807)
I20260812 21:03:56.768337 22328 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.254.130@KRBTEST.COM'} at 127.11.254.130:53097
WARNING: no policy specified for kudu/127.11.254.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.254.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.254.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.131.keytab.
Entry for principal kudu/127.11.254.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.131.keytab.
I20260812 21:03:56.793334 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568607354921-12282-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568607354921-12282-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568607354921-12282-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568607354921-12282-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.254.131:0
--local_ip_for_outbound_sockets=127.11.254.131
--webserver_interface=127.11.254.131
--webserver_port=0
--tserver_master_addrs=127.11.254.190:38587
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.131.keytab
--principal=kudu/127.11.254.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.254.148:40169
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568607354921-12282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568607354921-12282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568607354921-12282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:03:56.877933 22893 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:03:56.878186 22893 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:03:56.878217 22893 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:03:56.879423 22893 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:03:56.879472 22893 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:03:56.879508 22893 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.254.131
I20260812 21:03:56.881027 22893 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.254.148:40169
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568607354921-12282-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568607354921-12282-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.131.keytab
--principal=kudu/127.11.254.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.254.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568607354921-12282-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.254.131
--webserver_port=0
--tserver_master_addrs=127.11.254.190:38587
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22893
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.254.131
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568607354921-12282-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:03:56.881266 22893 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:03:56.881486 22893 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:03:56.886561 22904 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:03:56.886544 22907 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:03:56.886539 22903 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:03:56.886662 22893 server_base.cc:1034] running on GCE node
I20260812 21:03:56.886982 22893 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:03:56.887240 22893 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:03:56.888422 22893 hybrid_clock.cc:648] HybridClock initialized: now 1786568636888406 us; error 31 us; skew 500 ppm
Aug 12 21:03:56 dist-test-slave-vpvm krb5kdc[22245](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568636, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:03:56.891155 22893 init.cc:377] Logged in from keytab as kudu/127.11.254.131@KRBTEST.COM (short username kudu)
I20260812 21:03:56.891690 22893 webserver.cc:460] Webserver started at http://127.11.254.131:41341/ using document root <none> and password file <none>
I20260812 21:03:56.891899 22893 fs_manager.cc:359] Metadata directory not provided
I20260812 21:03:56.891944 22893 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:03:56.892078 22893 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:03:56.893154 22893 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568607354921-12282-0/minicluster-data/ts-2/data/instance:
uuid: "672f7c9470fb4585b7363e334a946683"
format_stamp: "Formatted at 2026-08-12 21:03:56 on dist-test-slave-vpvm"
server_key: "6957cd3db9f5e918205300b96e1c7372"
server_key_iv: "808e0c4d632f738d0b79701b31a014db"
server_key_version: "encryptionkey@0"
I20260812 21:03:56.893501 22893 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568607354921-12282-0/minicluster-data/ts-2/wal/instance:
uuid: "672f7c9470fb4585b7363e334a946683"
format_stamp: "Formatted at 2026-08-12 21:03:56 on dist-test-slave-vpvm"
server_key: "6957cd3db9f5e918205300b96e1c7372"
server_key_iv: "808e0c4d632f738d0b79701b31a014db"
server_key_version: "encryptionkey@0"
I20260812 21:03:56.894870 22893 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 21:03:56.895617 22919 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:03:56.895820 22893 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 21:03:56.895912 22893 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568607354921-12282-0/minicluster-data/ts-2/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568607354921-12282-0/minicluster-data/ts-2/wal
uuid: "672f7c9470fb4585b7363e334a946683"
format_stamp: "Formatted at 2026-08-12 21:03:56 on dist-test-slave-vpvm"
server_key: "6957cd3db9f5e918205300b96e1c7372"
server_key_iv: "808e0c4d632f738d0b79701b31a014db"
server_key_version: "encryptionkey@0"
I20260812 21:03:56.895979 22893 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568607354921-12282-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568607354921-12282-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568607354921-12282-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:03:56.903153 22893 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:03:56.905232 22893 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:03:56.905372 22893 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:03:56.905607 22893 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:03:56.905910 22893 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:03:56.905943 22893 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:03:56.905992 22893 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:03:56.906023 22893 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:03:56.911860 22893 rpc_server.cc:307] RPC server started. Bound to: 127.11.254.131:45389
I20260812 21:03:56.911880 23105 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.254.131:45389 every 8 connection(s)
I20260812 21:03:56.912240 22893 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568607354921-12282-0/minicluster-data/ts-2/data/info.pb
Aug 12 21:03:56 dist-test-slave-vpvm krb5kdc[22245](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568636, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.131@KRBTEST.COM for kudu/127.11.254.190@KRBTEST.COM
I20260812 21:03:56.919196 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 22893
I20260812 21:03:56.919304 12282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568607354921-12282-0/minicluster-data/ts-2/wal/instance
I20260812 21:03:56.919564 12282 external_mini_cluster.cc:1428] Setting key 437de71793dfc3320a792a9344365958
I20260812 21:03:56.922252 22389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:56.913012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.131:60899 (local address 127.11.254.190:38587)
0812 21:03:56.913070 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0812 21:03:56.913072 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:03:56.913315 (+ 243us) server_negotiation.cc:407] Connection header received
0812 21:03:56.914287 (+ 972us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:03:56.914288 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:03:56.914301 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:03:56.914364 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:03:56.915033 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:56.915561 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:56.916808 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:56.917013 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:56.917081 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:56.919491 (+ 2410us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:03:56.919492 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:03:56.919493 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:03:56.919503 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:03:56.921020 (+ 1517us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:56.921434 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:56.921434 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:56.921435 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:56.921487 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:56.921669 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:56.921669 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:56.921669 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:56.921784 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:03:56.921830 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0812 21:03:56.922097 (+ 267us) server_negotiation.cc:299] Negotiation successful
0812 21:03:56.922185 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260812 21:03:56.922523 23114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:56.912996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:38587 (local address 127.11.254.131:60899)
0812 21:03:56.913164 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0812 21:03:56.913209 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 21:03:56.914195 (+ 986us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:03:56.914435 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:03:56.914438 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:03:56.914709 (+ 271us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:03:56.914912 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:56.914917 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:56.915674 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:56.915676 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:56.916617 (+ 941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:56.916620 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:56.917120 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:56.917121 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:56.917245 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:56.917770 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:03:56.917779 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:03:56.919377 (+ 1598us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:03:56.921105 (+ 1728us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:03:56.921112 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:03:56.921114 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:03:56.921374 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:03:56.921528 (+ 154us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:03:56.921528 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:03:56.921528 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:03:56.921622 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:03:56.921830 (+ 208us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:03:56.921833 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:03:56.921987 (+ 154us) client_negotiation.cc:769] Sending connection context
0812 21:03:56.922083 (+ 96us) client_negotiation.cc:240] Negotiation successful
0812 21:03:56.922245 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":77,"threads_started":1}
I20260812 21:03:56.923014 23106 heartbeater.cc:347] Connected to a master server at 127.11.254.190:38587
I20260812 21:03:56.923110 23106 heartbeater.cc:464] Registering TS with master...
I20260812 21:03:56.923291 23106 heartbeater.cc:511] Master 127.11.254.190:38587 requested a full tablet report, sending...
I20260812 21:03:56.923632 22328 ts_manager.cc:194] Registered new tserver with Master: 672f7c9470fb4585b7363e334a946683 (127.11.254.131:45389)
I20260812 21:03:56.923995 22328 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.254.131@KRBTEST.COM'} at 127.11.254.131:60899
I20260812 21:03:56.931710 12282 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568607354921-12282-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568607354921-12282-0/krb5kdc/test-admin.keytab.
Aug 12 21:03:56 dist-test-slave-vpvm krb5kdc[22245](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568636, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:03:56.950760 12282 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260812 21:03:56.952582 23129 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:03:56.951685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53810 (local address 127.11.254.129:34329)
0812 21:03:56.951862 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0812 21:03:56.951867 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 21:03:56.951878 (+ 11us) server_negotiation.cc:407] Connection header received
0812 21:03:56.951906 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:03:56.951908 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:03:56.951923 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:03:56.951989 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:03:56.952491 (+ 502us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53810: BlockingRecv error: recv got EOF from 127.0.0.1:53810 (error 108)
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":94,"threads_started":1}
W20260812 21:03:56.952646 12282 init.cc:244] Error while opening credential cache '/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568607354921-12282-0/krb5kdc/krb5cc' of type FILE: Runtime error: Bad format in credentials cache (filename: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568607354921-12282-0/krb5kdc/krb5cc)
Aug 12 21:03:56 dist-test-slave-vpvm krb5kdc[22245](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568636, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:03:56.953733 12282 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 12 21:03:56 dist-test-slave-vpvm krb5kdc[22245](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568636, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:03:56.954783 12282 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 12 21:03:56 dist-test-slave-vpvm krb5kdc[22245](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568636, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.254.129@KRBTEST.COM
I20260812 21:03:56.962896 23129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:56.955423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53822 (local address 127.11.254.129:34329)
0812 21:03:56.955475 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0812 21:03:56.955477 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:03:56.955487 (+ 10us) server_negotiation.cc:407] Connection header received
0812 21:03:56.955550 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:03:56.955551 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:03:56.955562 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:03:56.955601 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:03:56.955982 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:56.956738 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:56.957473 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:56.958032 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:56.958099 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:56.960235 (+ 2136us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:03:56.960244 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:03:56.960244 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:03:56.960252 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:03:56.961754 (+ 1502us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:56.962154 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:56.962157 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:56.962157 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:56.962216 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:56.962428 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:56.962428 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:56.962428 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:56.962564 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:03:56.962681 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0812 21:03:56.962753 (+ 72us) server_negotiation.cc:299] Negotiation successful
0812 21:03:56.962820 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
W20260812 21:03:56.964726 23129 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:03:56.964230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53830 (local address 127.11.254.129:34329)
0812 21:03:56.964286 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0812 21:03:56.964288 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:03:56.964296 (+ 8us) server_negotiation.cc:407] Connection header received
0812 21:03:56.964346 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:03:56.964347 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:03:56.964358 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:03:56.964409 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:03:56.964676 (+ 267us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53830: BlockingRecv error: recv got EOF from 127.0.0.1:53830 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
W20260812 21:03:56.964906 12282 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568607354921-12282-0/krb5kdc/krb5cc' of type FILE
Aug 12 21:03:56 dist-test-slave-vpvm krb5kdc[22245](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568636, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:03:56.966034 12282 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 12 21:03:56 dist-test-slave-vpvm krb5kdc[22245](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568636, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:03:56.966912 12282 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 12 21:03:56 dist-test-slave-vpvm krb5kdc[22245](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568636, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.254.129@KRBTEST.COM
I20260812 21:03:56.973521 23129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:56.967395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53846 (local address 127.11.254.129:34329)
0812 21:03:56.967538 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0812 21:03:56.967540 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:03:56.967548 (+ 8us) server_negotiation.cc:407] Connection header received
0812 21:03:56.967576 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:03:56.967577 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:03:56.967587 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:03:56.967638 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:03:56.967950 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:56.968481 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:56.969117 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:56.969306 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:56.969348 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:56.971305 (+ 1957us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:03:56.971307 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:03:56.971307 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:03:56.971314 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:03:56.972580 (+ 1266us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:56.972882 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:56.972883 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:56.972883 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:56.972937 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:56.973113 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:56.973114 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:56.973114 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:56.973202 (+ 88us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:03:56.973266 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0812 21:03:56.973342 (+ 76us) server_negotiation.cc:299] Negotiation successful
0812 21:03:56.973399 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119}
W20260812 21:03:56.974908 23129 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:03:56.974491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53850 (local address 127.11.254.129:34329)
0812 21:03:56.974528 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0812 21:03:56.974530 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:03:56.974571 (+ 41us) server_negotiation.cc:407] Connection header received
0812 21:03:56.974617 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:03:56.974618 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:03:56.974625 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:03:56.974664 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:03:56.974864 (+ 200us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53850: BlockingRecv error: recv got EOF from 127.0.0.1:53850 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260812 21:03:56.975025 12282 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786568607354921-12282-0/krb5kdc/krb5cc' of type FILE
Aug 12 21:03:56 dist-test-slave-vpvm krb5kdc[22245](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568636, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:03:56.975944 12282 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 12 21:03:56 dist-test-slave-vpvm krb5kdc[22245](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568636, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:03:56.976706 12282 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 12 21:03:56 dist-test-slave-vpvm krb5kdc[22245](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568636, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.254.129@KRBTEST.COM
I20260812 21:03:56.983415 23129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:56.977187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53858 (local address 127.11.254.129:34329)
0812 21:03:56.977253 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0812 21:03:56.977255 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:03:56.977264 (+ 9us) server_negotiation.cc:407] Connection header received
0812 21:03:56.977316 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:03:56.977316 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:03:56.977324 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:03:56.977367 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:03:56.977704 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:56.978255 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:56.978894 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:56.979083 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:56.979133 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:56.981102 (+ 1969us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:03:56.981104 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:03:56.981104 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:03:56.981116 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:03:56.982465 (+ 1349us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:56.982755 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:56.982756 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:56.982756 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:56.982808 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:56.982938 (+ 130us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:56.982939 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:56.982939 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:56.983032 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:03:56.983098 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0812 21:03:56.983270 (+ 172us) server_negotiation.cc:299] Negotiation successful
0812 21:03:56.983326 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 21:03:56.984148 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 22435
I20260812 21:03:56.997649 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 22672
I20260812 21:03:57.009307 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 22893
I20260812 21:03:57.020277 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 22271
2026-08-12T21:03:57Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2094 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 21:03:57 dist-test-slave-vpvm krb5kdc[23147](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 21:03:57 dist-test-slave-vpvm krb5kdc[23147](info): set up 2 sockets
Aug 12 21:03:57 dist-test-slave-vpvm krb5kdc[23147](info): commencing operation
krb5kdc: starting...
W20260812 21:03:58.204056 12282 mini_kdc.cc:121] Time spent starting KDC: real 1.154s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/krb5kdc/test-user.keytab.
Aug 12 21:03:58 dist-test-slave-vpvm krb5kdc[23147](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568638, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T21:03:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T21:03:58Z Disabled control of system clock
WARNING: no policy specified for oryx/127.11.254.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.254.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.254.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/krb5kdc/oryx_127.11.254.190.keytab.
Entry for principal oryx/127.11.254.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/krb5kdc/oryx_127.11.254.190.keytab.
I20260812 21:03:58.331099 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.254.190:43007
--webserver_interface=127.11.254.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/krb5kdc/oryx_127.11.254.190.keytab
--principal=oryx/127.11.254.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.254.148:45807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.254.190:43007
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:03:58.414245 23167 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:03:58.414462 23167 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:03:58.414494 23167 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:03:58.415691 23167 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 21:03:58.415737 23167 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:03:58.415755 23167 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:03:58.415772 23167 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 21:03:58.415787 23167 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 21:03:58.417261 23167 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.254.148:45807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.254.190:43007
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/krb5kdc/oryx_127.11.254.190.keytab
--principal=oryx/127.11.254.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.254.190:43007
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.254.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23167
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:03:58.417515 23167 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:03:58.417719 23167 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:03:58.422571 23181 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:03:58.422580 23183 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:03:58.422612 23176 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:03:58.422784 23167 server_base.cc:1034] running on GCE node
I20260812 21:03:58.423098 23167 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:03:58.423368 23167 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:03:58.424518 23167 hybrid_clock.cc:648] HybridClock initialized: now 1786568638424501 us; error 32 us; skew 500 ppm
Aug 12 21:03:58 dist-test-slave-vpvm krb5kdc[23147](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568638, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.254.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:03:58.426981 23167 init.cc:377] Logged in from keytab as oryx/127.11.254.190@KRBTEST.COM (short username oryx)
I20260812 21:03:58.427415 23167 webserver.cc:460] Webserver started at http://127.11.254.190:43433/ using document root <none> and password file <none>
I20260812 21:03:58.427601 23167 fs_manager.cc:359] Metadata directory not provided
I20260812 21:03:58.427640 23167 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:03:58.427733 23167 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:03:58.428771 23167 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/master-0/data/instance:
uuid: "c90e0f61a5354abbbf96692370a65ebd"
format_stamp: "Formatted at 2026-08-12 21:03:58 on dist-test-slave-vpvm"
server_key: "3605f1aaa4ccab2ed110c6236a7b639d"
server_key_iv: "9545c52f5ed488bbfb7cc5189bef6f29"
server_key_version: "encryptionkey@0"
I20260812 21:03:58.429096 23167 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/master-0/wal/instance:
uuid: "c90e0f61a5354abbbf96692370a65ebd"
format_stamp: "Formatted at 2026-08-12 21:03:58 on dist-test-slave-vpvm"
server_key: "3605f1aaa4ccab2ed110c6236a7b639d"
server_key_iv: "9545c52f5ed488bbfb7cc5189bef6f29"
server_key_version: "encryptionkey@0"
I20260812 21:03:58.430372 23167 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260812 21:03:58.431056 23196 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:03:58.431213 23167 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 21:03:58.431288 23167 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/master-0/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/master-0/wal
uuid: "c90e0f61a5354abbbf96692370a65ebd"
format_stamp: "Formatted at 2026-08-12 21:03:58 on dist-test-slave-vpvm"
server_key: "3605f1aaa4ccab2ed110c6236a7b639d"
server_key_iv: "9545c52f5ed488bbfb7cc5189bef6f29"
server_key_version: "encryptionkey@0"
I20260812 21:03:58.431357 23167 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:03:58.447052 23167 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:03:58.449272 23167 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:03:58.449412 23167 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:03:58.453213 23167 rpc_server.cc:307] RPC server started. Bound to: 127.11.254.190:43007
I20260812 21:03:58.453275 23279 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.254.190:43007 every 8 connection(s)
I20260812 21:03:58.453596 23167 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
I20260812 21:03:58.454602 23280 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:03:58.456284 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 23167
I20260812 21:03:58.456395 12282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/master-0/wal/instance
I20260812 21:03:58.456566 12282 external_mini_cluster.cc:1428] Setting key 1c2fdb808ee68104fb3aec09405149b7
I20260812 21:03:58.457101 23280 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c90e0f61a5354abbbf96692370a65ebd: Bootstrap starting.
I20260812 21:03:58.457732 23280 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c90e0f61a5354abbbf96692370a65ebd: Neither blocks nor log segments found. Creating new log.
I20260812 21:03:58.458058 23280 log.cc:824] T 00000000000000000000000000000000 P c90e0f61a5354abbbf96692370a65ebd: Log is configured to *not* fsync() on all Append() calls
I20260812 21:03:58.458823 23280 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c90e0f61a5354abbbf96692370a65ebd: No bootstrap required, opened a new log
I20260812 21:03:58.460891 23280 raft_consensus.cc:348] T 00000000000000000000000000000000 P c90e0f61a5354abbbf96692370a65ebd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c90e0f61a5354abbbf96692370a65ebd" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 43007 } }
I20260812 21:03:58.461032 23280 raft_consensus.cc:374] T 00000000000000000000000000000000 P c90e0f61a5354abbbf96692370a65ebd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:03:58.461053 23280 raft_consensus.cc:729] T 00000000000000000000000000000000 P c90e0f61a5354abbbf96692370a65ebd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c90e0f61a5354abbbf96692370a65ebd, State: Initialized, Role: FOLLOWER
I20260812 21:03:58.461160 23280 consensus_queue.cc:260] T 00000000000000000000000000000000 P c90e0f61a5354abbbf96692370a65ebd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c90e0f61a5354abbbf96692370a65ebd" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 43007 } }
I20260812 21:03:58.461246 23280 raft_consensus.cc:388] T 00000000000000000000000000000000 P c90e0f61a5354abbbf96692370a65ebd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 21:03:58.461273 23280 raft_consensus.cc:482] T 00000000000000000000000000000000 P c90e0f61a5354abbbf96692370a65ebd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 21:03:58.461320 23280 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c90e0f61a5354abbbf96692370a65ebd [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:03:58.461875 23280 raft_consensus.cc:504] T 00000000000000000000000000000000 P c90e0f61a5354abbbf96692370a65ebd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c90e0f61a5354abbbf96692370a65ebd" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 43007 } }
I20260812 21:03:58.462001 23280 leader_election.cc:302] T 00000000000000000000000000000000 P c90e0f61a5354abbbf96692370a65ebd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c90e0f61a5354abbbf96692370a65ebd; no voters:
I20260812 21:03:58.462162 23280 leader_election.cc:288] T 00000000000000000000000000000000 P c90e0f61a5354abbbf96692370a65ebd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 21:03:58.462195 23286 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c90e0f61a5354abbbf96692370a65ebd [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:03:58.462316 23286 raft_consensus.cc:686] T 00000000000000000000000000000000 P c90e0f61a5354abbbf96692370a65ebd [term 1 LEADER]: Becoming Leader. State: Replica: c90e0f61a5354abbbf96692370a65ebd, State: Running, Role: LEADER
I20260812 21:03:58.462401 23286 consensus_queue.cc:237] T 00000000000000000000000000000000 P c90e0f61a5354abbbf96692370a65ebd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c90e0f61a5354abbbf96692370a65ebd" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 43007 } }
I20260812 21:03:58.462436 23280 sys_catalog.cc:564] T 00000000000000000000000000000000 P c90e0f61a5354abbbf96692370a65ebd [sys.catalog]: configured and running, proceeding with master startup.
I20260812 21:03:58.462733 23286 sys_catalog.cc:455] T 00000000000000000000000000000000 P c90e0f61a5354abbbf96692370a65ebd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c90e0f61a5354abbbf96692370a65ebd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c90e0f61a5354abbbf96692370a65ebd" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 43007 } } }
I20260812 21:03:58.462805 23286 sys_catalog.cc:458] T 00000000000000000000000000000000 P c90e0f61a5354abbbf96692370a65ebd [sys.catalog]: This master's current role is: LEADER
I20260812 21:03:58.462920 23286 sys_catalog.cc:455] T 00000000000000000000000000000000 P c90e0f61a5354abbbf96692370a65ebd [sys.catalog]: SysCatalogTable state changed. Reason: New leader c90e0f61a5354abbbf96692370a65ebd. Latest consensus state: current_term: 1 leader_uuid: "c90e0f61a5354abbbf96692370a65ebd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c90e0f61a5354abbbf96692370a65ebd" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 43007 } } }
I20260812 21:03:58.462977 23286 sys_catalog.cc:458] T 00000000000000000000000000000000 P c90e0f61a5354abbbf96692370a65ebd [sys.catalog]: This master's current role is: LEADER
I20260812 21:03:58.463093 23296 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Aug 12 21:03:58 dist-test-slave-vpvm krb5kdc[23147](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568638, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.11.254.190@KRBTEST.COM
I20260812 21:03:58.463642 23296 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 21:03:58.467394 23283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:58.457074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48402 (local address 127.11.254.190:43007)
0812 21:03:58.457284 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0812 21:03:58.457292 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 21:03:58.457317 (+ 25us) server_negotiation.cc:407] Connection header received
0812 21:03:58.457804 (+ 487us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:03:58.457813 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:03:58.457883 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:03:58.458040 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:03:58.458755 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:58.459712 (+ 957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:58.460805 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:58.461285 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:58.461389 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:58.463873 (+ 2484us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:03:58.463877 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:03:58.463877 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:03:58.463889 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:03:58.465781 (+ 1892us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:58.466221 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:58.466224 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:58.466225 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:58.466339 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:58.466506 (+ 167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:58.466507 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:58.466507 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:58.466824 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:03:58.466977 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0812 21:03:58.467046 (+ 69us) server_negotiation.cc:299] Negotiation successful
0812 21:03:58.467192 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":73,"threads_started":1}
I20260812 21:03:58.470717 23296 catalog_manager.cc:1347] Generated new cluster ID: b39e89298b2f46b5a2e07cc9a4507645
I20260812 21:03:58.470780 23296 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 21:03:58.478322 23296 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 21:03:58.479079 23296 catalog_manager.cc:1504] Loading token signing keys...
I20260812 21:03:58.484577 23296 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c90e0f61a5354abbbf96692370a65ebd: Generated new TSK 0
I20260812 21:03:58.484812 23296 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.11.254.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.254.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.254.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/krb5kdc/oryx_127.11.254.129.keytab.
Entry for principal oryx/127.11.254.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/krb5kdc/oryx_127.11.254.129.keytab.
I20260812 21:03:58.514004 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.254.129:0
--local_ip_for_outbound_sockets=127.11.254.129
--webserver_interface=127.11.254.129
--webserver_port=0
--tserver_master_addrs=127.11.254.190:43007
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/krb5kdc/oryx_127.11.254.129.keytab
--principal=oryx/127.11.254.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.254.148:45807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:03:58.598426 23324 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:03:58.598654 23324 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:03:58.598688 23324 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:03:58.599890 23324 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:03:58.599936 23324 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:03:58.599972 23324 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.254.129
I20260812 21:03:58.601475 23324 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.254.148:45807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/krb5kdc/oryx_127.11.254.129.keytab
--principal=oryx/127.11.254.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.254.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.254.129
--webserver_port=0
--tserver_master_addrs=127.11.254.190:43007
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23324
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.254.129
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:03:58.601722 23324 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:03:58.601969 23324 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:03:58.607132 23334 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:03:58.607132 23335 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:03:58.607211 23337 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:03:58.607498 23324 server_base.cc:1034] running on GCE node
I20260812 21:03:58.607725 23324 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:03:58.607965 23324 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:03:58.609107 23324 hybrid_clock.cc:648] HybridClock initialized: now 1786568638609093 us; error 34 us; skew 500 ppm
Aug 12 21:03:58 dist-test-slave-vpvm krb5kdc[23147](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568638, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.254.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:03:58.611716 23324 init.cc:377] Logged in from keytab as oryx/127.11.254.129@KRBTEST.COM (short username oryx)
I20260812 21:03:58.612160 23324 webserver.cc:460] Webserver started at http://127.11.254.129:37093/ using document root <none> and password file <none>
I20260812 21:03:58.612371 23324 fs_manager.cc:359] Metadata directory not provided
I20260812 21:03:58.612419 23324 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:03:58.612545 23324 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:03:58.613602 23324 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/ts-0/data/instance:
uuid: "8e16af0efe6c4f698e93d11250de344b"
format_stamp: "Formatted at 2026-08-12 21:03:58 on dist-test-slave-vpvm"
server_key: "5627151f9d2ca36ca770267088374f4c"
server_key_iv: "2de650a821e575b652c94dc69933702d"
server_key_version: "encryptionkey@0"
I20260812 21:03:58.613988 23324 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/ts-0/wal/instance:
uuid: "8e16af0efe6c4f698e93d11250de344b"
format_stamp: "Formatted at 2026-08-12 21:03:58 on dist-test-slave-vpvm"
server_key: "5627151f9d2ca36ca770267088374f4c"
server_key_iv: "2de650a821e575b652c94dc69933702d"
server_key_version: "encryptionkey@0"
I20260812 21:03:58.615262 23324 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 21:03:58.615957 23348 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:03:58.616107 23324 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 21:03:58.616201 23324 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/ts-0/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/ts-0/wal
uuid: "8e16af0efe6c4f698e93d11250de344b"
format_stamp: "Formatted at 2026-08-12 21:03:58 on dist-test-slave-vpvm"
server_key: "5627151f9d2ca36ca770267088374f4c"
server_key_iv: "2de650a821e575b652c94dc69933702d"
server_key_version: "encryptionkey@0"
I20260812 21:03:58.616268 23324 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:03:58.627482 23324 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:03:58.629417 23324 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:03:58.629570 23324 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:03:58.629827 23324 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:03:58.630205 23324 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:03:58.630242 23324 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:03:58.630273 23324 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:03:58.630291 23324 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:03:58.636227 23324 rpc_server.cc:307] RPC server started. Bound to: 127.11.254.129:35727
I20260812 21:03:58.636274 23534 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.254.129:35727 every 8 connection(s)
I20260812 21:03:58.636612 23324 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/ts-0/data/info.pb
I20260812 21:03:58.639966 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 23324
I20260812 21:03:58.640053 12282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/ts-0/wal/instance
I20260812 21:03:58.640230 12282 external_mini_cluster.cc:1428] Setting key 7c0d3f35b70689468d5a0c5aa21d6566
Aug 12 21:03:58 dist-test-slave-vpvm krb5kdc[23147](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568638, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.254.129@KRBTEST.COM for oryx/127.11.254.190@KRBTEST.COM
I20260812 21:03:58.646993 23283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:58.637346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.129:56141 (local address 127.11.254.190:43007)
0812 21:03:58.637414 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0812 21:03:58.637416 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:03:58.637612 (+ 196us) server_negotiation.cc:407] Connection header received
0812 21:03:58.638190 (+ 578us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:03:58.638191 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:03:58.638204 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:03:58.638263 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:03:58.639277 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:58.639813 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:58.641178 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:58.641437 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:58.641508 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:58.643987 (+ 2479us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:03:58.643989 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:03:58.643989 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:03:58.643997 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:03:58.645384 (+ 1387us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:58.645915 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:58.645916 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:58.645916 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:58.646032 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:58.646278 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:58.646279 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:58.646280 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:58.646434 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:03:58.646511 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0812 21:03:58.646780 (+ 269us) server_negotiation.cc:299] Negotiation successful
0812 21:03:58.646885 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260812 21:03:58.647048 23537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:58.637297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:43007 (local address 127.11.254.129:56141)
0812 21:03:58.637485 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0812 21:03:58.637524 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 21:03:58.638087 (+ 563us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:03:58.638338 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:03:58.638341 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:03:58.638969 (+ 628us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:03:58.639177 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:58.639183 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:58.639919 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:58.639920 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:58.641091 (+ 1171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:58.641093 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:58.641538 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:58.641539 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:58.641685 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:58.642212 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:03:58.642221 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:03:58.643860 (+ 1639us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:03:58.645510 (+ 1650us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:03:58.645519 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:03:58.645522 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:03:58.645830 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:03:58.646104 (+ 274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:03:58.646104 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:03:58.646104 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:03:58.646218 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:03:58.646495 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:03:58.646498 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:03:58.646648 (+ 150us) client_negotiation.cc:769] Sending connection context
0812 21:03:58.646732 (+ 84us) client_negotiation.cc:240] Negotiation successful
0812 21:03:58.646870 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":103,"threads_started":1}
I20260812 21:03:58.647710 23535 heartbeater.cc:347] Connected to a master server at 127.11.254.190:43007
I20260812 21:03:58.647812 23535 heartbeater.cc:464] Registering TS with master...
I20260812 21:03:58.648000 23535 heartbeater.cc:511] Master 127.11.254.190:43007 requested a full tablet report, sending...
I20260812 21:03:58.648568 23224 ts_manager.cc:194] Registered new tserver with Master: 8e16af0efe6c4f698e93d11250de344b (127.11.254.129:35727)
I20260812 21:03:58.649267 23224 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.11.254.129@KRBTEST.COM'} at 127.11.254.129:56141
WARNING: no policy specified for oryx/127.11.254.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.254.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.254.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/krb5kdc/oryx_127.11.254.130.keytab.
Entry for principal oryx/127.11.254.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/krb5kdc/oryx_127.11.254.130.keytab.
I20260812 21:03:58.669791 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.254.130:0
--local_ip_for_outbound_sockets=127.11.254.130
--webserver_interface=127.11.254.130
--webserver_port=0
--tserver_master_addrs=127.11.254.190:43007
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/krb5kdc/oryx_127.11.254.130.keytab
--principal=oryx/127.11.254.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.254.148:45807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:03:58.754463 23543 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:03:58.754702 23543 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:03:58.754727 23543 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:03:58.755909 23543 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:03:58.755957 23543 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:03:58.755983 23543 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.254.130
I20260812 21:03:58.757402 23543 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.254.148:45807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/krb5kdc/oryx_127.11.254.130.keytab
--principal=oryx/127.11.254.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.254.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.254.130
--webserver_port=0
--tserver_master_addrs=127.11.254.190:43007
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23543
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.254.130
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:03:58.757609 23543 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:03:58.757805 23543 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 21:03:58.762987 23543 server_base.cc:1034] running on GCE node
W20260812 21:03:58.762935 23553 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:03:58.762966 23552 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:03:58.762938 23557 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:03:58.763299 23543 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:03:58.763538 23543 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:03:58.764689 23543 hybrid_clock.cc:648] HybridClock initialized: now 1786568638764680 us; error 42 us; skew 500 ppm
Aug 12 21:03:58 dist-test-slave-vpvm krb5kdc[23147](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568638, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.254.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:03:58.767256 23543 init.cc:377] Logged in from keytab as oryx/127.11.254.130@KRBTEST.COM (short username oryx)
I20260812 21:03:58.767721 23543 webserver.cc:460] Webserver started at http://127.11.254.130:33467/ using document root <none> and password file <none>
I20260812 21:03:58.767922 23543 fs_manager.cc:359] Metadata directory not provided
I20260812 21:03:58.767966 23543 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:03:58.768090 23543 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:03:58.769153 23543 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/ts-1/data/instance:
uuid: "a9da4e76358f4ee18755f38b4d8de7ad"
format_stamp: "Formatted at 2026-08-12 21:03:58 on dist-test-slave-vpvm"
server_key: "4e7e5c2df263aff72b84962e259fd96c"
server_key_iv: "a1263a661ad10e300ff92c009df9e0d3"
server_key_version: "encryptionkey@0"
I20260812 21:03:58.769495 23543 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/ts-1/wal/instance:
uuid: "a9da4e76358f4ee18755f38b4d8de7ad"
format_stamp: "Formatted at 2026-08-12 21:03:58 on dist-test-slave-vpvm"
server_key: "4e7e5c2df263aff72b84962e259fd96c"
server_key_iv: "a1263a661ad10e300ff92c009df9e0d3"
server_key_version: "encryptionkey@0"
I20260812 21:03:58.770810 23543 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 21:03:58.771458 23571 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:03:58.771608 23543 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 21:03:58.771711 23543 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/ts-1/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/ts-1/wal
uuid: "a9da4e76358f4ee18755f38b4d8de7ad"
format_stamp: "Formatted at 2026-08-12 21:03:58 on dist-test-slave-vpvm"
server_key: "4e7e5c2df263aff72b84962e259fd96c"
server_key_iv: "a1263a661ad10e300ff92c009df9e0d3"
server_key_version: "encryptionkey@0"
I20260812 21:03:58.771767 23543 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:03:58.799301 23543 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:03:58.801352 23543 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:03:58.801481 23543 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:03:58.801709 23543 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:03:58.802054 23543 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:03:58.802091 23543 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:03:58.802115 23543 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:03:58.802136 23543 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:03:58.808045 23543 rpc_server.cc:307] RPC server started. Bound to: 127.11.254.130:39443
I20260812 21:03:58.808145 23776 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.254.130:39443 every 8 connection(s)
I20260812 21:03:58.808430 23543 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/ts-1/data/info.pb
Aug 12 21:03:58 dist-test-slave-vpvm krb5kdc[23147](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568638, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.254.130@KRBTEST.COM for oryx/127.11.254.190@KRBTEST.COM
I20260812 21:03:58.815873 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 23543
I20260812 21:03:58.815981 12282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/ts-1/wal/instance
I20260812 21:03:58.816150 12282 external_mini_cluster.cc:1428] Setting key 64547607d84985dd01aebc040fb5f346
I20260812 21:03:58.817834 23283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:58.809159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.130:53245 (local address 127.11.254.190:43007)
0812 21:03:58.809215 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0812 21:03:58.809217 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:03:58.809433 (+ 216us) server_negotiation.cc:407] Connection header received
0812 21:03:58.809916 (+ 483us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:03:58.809917 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:03:58.809928 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:03:58.810025 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:03:58.810969 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:58.811537 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:58.812514 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:58.812713 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:58.812760 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:58.814832 (+ 2072us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:03:58.814834 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:03:58.814834 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:03:58.814842 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:03:58.816409 (+ 1567us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:58.816846 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:58.816848 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:58.816848 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:58.816904 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:58.817153 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:58.817153 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:58.817154 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:58.817270 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:03:58.817331 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0812 21:03:58.817653 (+ 322us) server_negotiation.cc:299] Negotiation successful
0812 21:03:58.817738 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 21:03:58.818060 23783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:58.809122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:43007 (local address 127.11.254.130:53245)
0812 21:03:58.809308 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0812 21:03:58.809346 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 21:03:58.809834 (+ 488us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:03:58.810085 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:03:58.810088 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:03:58.810682 (+ 594us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:03:58.810893 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:58.810898 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:58.811613 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:58.811613 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:58.812432 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:58.812435 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:58.812771 (+ 336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:58.812772 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:58.812870 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:58.813319 (+ 449us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:03:58.813328 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:03:58.814735 (+ 1407us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:03:58.816534 (+ 1799us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:03:58.816542 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:03:58.816545 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:03:58.816777 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:03:58.816967 (+ 190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:03:58.816968 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:03:58.816968 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:03:58.817092 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:03:58.817374 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:03:58.817380 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:03:58.817526 (+ 146us) client_negotiation.cc:769] Sending connection context
0812 21:03:58.817619 (+ 93us) client_negotiation.cc:240] Negotiation successful
0812 21:03:58.817797 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":114,"threads_started":1}
I20260812 21:03:58.818528 23777 heartbeater.cc:347] Connected to a master server at 127.11.254.190:43007
I20260812 21:03:58.818629 23777 heartbeater.cc:464] Registering TS with master...
I20260812 21:03:58.818815 23777 heartbeater.cc:511] Master 127.11.254.190:43007 requested a full tablet report, sending...
I20260812 21:03:58.819224 23224 ts_manager.cc:194] Registered new tserver with Master: a9da4e76358f4ee18755f38b4d8de7ad (127.11.254.130:39443)
I20260812 21:03:58.819721 23224 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.11.254.130@KRBTEST.COM'} at 127.11.254.130:53245
WARNING: no policy specified for oryx/127.11.254.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.254.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.254.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/krb5kdc/oryx_127.11.254.131.keytab.
Entry for principal oryx/127.11.254.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/krb5kdc/oryx_127.11.254.131.keytab.
I20260812 21:03:58.844445 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.254.131:0
--local_ip_for_outbound_sockets=127.11.254.131
--webserver_interface=127.11.254.131
--webserver_port=0
--tserver_master_addrs=127.11.254.190:43007
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/krb5kdc/oryx_127.11.254.131.keytab
--principal=oryx/127.11.254.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.254.148:45807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:03:58.927243 23787 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:03:58.927470 23787 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:03:58.927495 23787 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:03:58.928637 23787 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:03:58.928680 23787 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:03:58.928707 23787 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.254.131
I20260812 21:03:58.930190 23787 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.254.148:45807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/krb5kdc/oryx_127.11.254.131.keytab
--principal=oryx/127.11.254.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.254.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.254.131
--webserver_port=0
--tserver_master_addrs=127.11.254.190:43007
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23787
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.254.131
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:03:58.930390 23787 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:03:58.930593 23787 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:03:58.935629 23798 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:03:58.935635 23795 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:03:58.935663 23793 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:03:58.936071 23787 server_base.cc:1034] running on GCE node
I20260812 21:03:58.936239 23787 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:03:58.936470 23787 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:03:58.937613 23787 hybrid_clock.cc:648] HybridClock initialized: now 1786568638937589 us; error 35 us; skew 500 ppm
Aug 12 21:03:58 dist-test-slave-vpvm krb5kdc[23147](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568638, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.254.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:03:58.940142 23787 init.cc:377] Logged in from keytab as oryx/127.11.254.131@KRBTEST.COM (short username oryx)
I20260812 21:03:58.940609 23787 webserver.cc:460] Webserver started at http://127.11.254.131:42619/ using document root <none> and password file <none>
I20260812 21:03:58.940830 23787 fs_manager.cc:359] Metadata directory not provided
I20260812 21:03:58.940881 23787 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:03:58.941013 23787 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:03:58.942170 23787 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/ts-2/data/instance:
uuid: "228262922b09413f8fe37401b0a932bd"
format_stamp: "Formatted at 2026-08-12 21:03:58 on dist-test-slave-vpvm"
server_key: "6ee62bd1ea25c42aecbcefc56e9fd6bc"
server_key_iv: "8674148ee857a2477fe937dd36369f62"
server_key_version: "encryptionkey@0"
I20260812 21:03:58.942535 23787 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/ts-2/wal/instance:
uuid: "228262922b09413f8fe37401b0a932bd"
format_stamp: "Formatted at 2026-08-12 21:03:58 on dist-test-slave-vpvm"
server_key: "6ee62bd1ea25c42aecbcefc56e9fd6bc"
server_key_iv: "8674148ee857a2477fe937dd36369f62"
server_key_version: "encryptionkey@0"
I20260812 21:03:58.943954 23787 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 21:03:58.944660 23808 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:03:58.944902 23787 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 21:03:58.944993 23787 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/ts-2/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/ts-2/wal
uuid: "228262922b09413f8fe37401b0a932bd"
format_stamp: "Formatted at 2026-08-12 21:03:58 on dist-test-slave-vpvm"
server_key: "6ee62bd1ea25c42aecbcefc56e9fd6bc"
server_key_iv: "8674148ee857a2477fe937dd36369f62"
server_key_version: "encryptionkey@0"
I20260812 21:03:58.945057 23787 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:03:58.955935 23787 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:03:58.957748 23787 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:03:58.957906 23787 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:03:58.958174 23787 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:03:58.958490 23787 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:03:58.958523 23787 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:03:58.958554 23787 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:03:58.958572 23787 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:03:58.964382 23787 rpc_server.cc:307] RPC server started. Bound to: 127.11.254.131:46307
I20260812 21:03:58.964452 24008 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.254.131:46307 every 8 connection(s)
I20260812 21:03:58.964775 23787 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/ts-2/data/info.pb
I20260812 21:03:58.970264 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 23787
I20260812 21:03:58.970345 12282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/ts-2/wal/instance
I20260812 21:03:58.970492 12282 external_mini_cluster.cc:1428] Setting key 44cc01fbc00fee00c696c5ef44b5fc96
Aug 12 21:03:58 dist-test-slave-vpvm krb5kdc[23147](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568638, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.254.131@KRBTEST.COM for oryx/127.11.254.190@KRBTEST.COM
I20260812 21:03:58.974088 23283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:58.965493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.131:49309 (local address 127.11.254.190:43007)
0812 21:03:58.965556 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0812 21:03:58.965558 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:03:58.965758 (+ 200us) server_negotiation.cc:407] Connection header received
0812 21:03:58.966328 (+ 570us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:03:58.966330 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:03:58.966342 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:03:58.966404 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:03:58.967350 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:58.967922 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:58.969058 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:58.969256 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:58.969318 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:58.971656 (+ 2338us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:03:58.971663 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:03:58.971663 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:03:58.971671 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:03:58.972965 (+ 1294us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:58.973322 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:58.973323 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:58.973323 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:58.973373 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:58.973581 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:58.973581 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:58.973581 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:58.973686 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:03:58.973734 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0812 21:03:58.973939 (+ 205us) server_negotiation.cc:299] Negotiation successful
0812 21:03:58.974021 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260812 21:03:58.974287 24016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:58.965447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:43007 (local address 127.11.254.131:49309)
0812 21:03:58.965607 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0812 21:03:58.965651 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 21:03:58.966229 (+ 578us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:03:58.966468 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:03:58.966471 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:03:58.967048 (+ 577us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:03:58.967265 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:58.967270 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:58.967998 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:58.967999 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:58.968944 (+ 945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:58.968946 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:58.969332 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:58.969333 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:58.969437 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:58.969914 (+ 477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:03:58.969922 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:03:58.971556 (+ 1634us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:03:58.973042 (+ 1486us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:03:58.973048 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:03:58.973049 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:03:58.973259 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:03:58.973426 (+ 167us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:03:58.973427 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:03:58.973427 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:03:58.973532 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:03:58.973731 (+ 199us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:03:58.973734 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:03:58.973846 (+ 112us) client_negotiation.cc:769] Sending connection context
0812 21:03:58.973918 (+ 72us) client_negotiation.cc:240] Negotiation successful
0812 21:03:58.974087 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":72,"threads_started":1}
I20260812 21:03:58.974766 24009 heartbeater.cc:347] Connected to a master server at 127.11.254.190:43007
I20260812 21:03:58.974854 24009 heartbeater.cc:464] Registering TS with master...
I20260812 21:03:58.975040 24009 heartbeater.cc:511] Master 127.11.254.190:43007 requested a full tablet report, sending...
I20260812 21:03:58.975376 23224 ts_manager.cc:194] Registered new tserver with Master: 228262922b09413f8fe37401b0a932bd (127.11.254.131:46307)
I20260812 21:03:58.975773 23224 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.11.254.131@KRBTEST.COM'} at 127.11.254.131:49309
I20260812 21:03:58.982306 12282 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.11.254.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.254.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.254.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/krb5kdc/oryx_127.11.254.189.keytab.
Entry for principal oryx/127.11.254.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/krb5kdc/oryx_127.11.254.189.keytab.
I20260812 21:03:59.009128 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.254.189:36155
--webserver_interface=127.11.254.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/krb5kdc/oryx_127.11.254.189.keytab
--principal=oryx/127.11.254.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.254.148:45807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.254.190:43007,127.11.254.189:36155
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:03:59.093604 24025 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:03:59.093837 24025 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:03:59.093870 24025 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:03:59.095111 24025 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 21:03:59.095165 24025 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:03:59.095184 24025 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:03:59.095201 24025 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 21:03:59.095216 24025 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 21:03:59.096732 24025 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.254.148:45807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.11.254.190:43007,127.11.254.189:36155
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/krb5kdc/oryx_127.11.254.189.keytab
--principal=oryx/127.11.254.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.254.189:36155
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.11.254.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24025
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
W20260812 21:03:59.096859 24025 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.11.254.190:43007,127.11.254.189:36155'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260812 21:03:59.097011 24025 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:03:59.097225 24025 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:03:59.102072 24034 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:03:59.102046 24036 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:03:59.102109 24033 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:03:59.102195 24025 server_base.cc:1034] running on GCE node
I20260812 21:03:59.102437 24025 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:03:59.102722 24025 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:03:59.103871 24025 hybrid_clock.cc:648] HybridClock initialized: now 1786568639103857 us; error 29 us; skew 500 ppm
Aug 12 21:03:59 dist-test-slave-vpvm krb5kdc[23147](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568639, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.254.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:03:59.106448 24025 init.cc:377] Logged in from keytab as oryx/127.11.254.189@KRBTEST.COM (short username oryx)
I20260812 21:03:59.107040 24025 webserver.cc:460] Webserver started at http://127.11.254.189:44387/ using document root <none> and password file <none>
I20260812 21:03:59.107287 24025 fs_manager.cc:359] Metadata directory not provided
I20260812 21:03:59.107339 24025 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:03:59.107471 24025 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:03:59.108561 24025 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/master-1/data/instance:
uuid: "5026b3627f2e49f8b4e7cf1521420cd1"
format_stamp: "Formatted at 2026-08-12 21:03:59 on dist-test-slave-vpvm"
server_key: "6ec19b8e883d1b3ea2e279d8f97038c7"
server_key_iv: "ee27801f1e0dc85f289f2b3d5ff6acdb"
server_key_version: "encryptionkey@0"
I20260812 21:03:59.108912 24025 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/master-1/wal/instance:
uuid: "5026b3627f2e49f8b4e7cf1521420cd1"
format_stamp: "Formatted at 2026-08-12 21:03:59 on dist-test-slave-vpvm"
server_key: "6ec19b8e883d1b3ea2e279d8f97038c7"
server_key_iv: "ee27801f1e0dc85f289f2b3d5ff6acdb"
server_key_version: "encryptionkey@0"
I20260812 21:03:59.110248 24025 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260812 21:03:59.111003 24046 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:03:59.111182 24025 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 21:03:59.111270 24025 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/master-1/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/master-1/wal
uuid: "5026b3627f2e49f8b4e7cf1521420cd1"
format_stamp: "Formatted at 2026-08-12 21:03:59 on dist-test-slave-vpvm"
server_key: "6ec19b8e883d1b3ea2e279d8f97038c7"
server_key_iv: "ee27801f1e0dc85f289f2b3d5ff6acdb"
server_key_version: "encryptionkey@0"
I20260812 21:03:59.111334 24025 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:03:59.132093 24025 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:03:59.134100 24025 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:03:59.134264 24025 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:03:59.138327 24025 rpc_server.cc:307] RPC server started. Bound to: 127.11.254.189:36155
I20260812 21:03:59.138373 24125 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.254.189:36155 every 8 connection(s)
I20260812 21:03:59.138720 24025 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/master-1/data/info.pb
I20260812 21:03:59.139698 24126 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:03:59.141299 24126 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.254.190" port: 43007 } has no permanent_uuid. Determining permanent_uuid...
I20260812 21:03:59.145385 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 24025
Aug 12 21:03:59 dist-test-slave-vpvm krb5kdc[23147](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568639, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.254.189@KRBTEST.COM for oryx/127.11.254.190@KRBTEST.COM
I20260812 21:03:59.150926 23283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:59.142058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38712 (local address 127.11.254.190:43007)
0812 21:03:59.142118 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0812 21:03:59.142119 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 21:03:59.142347 (+ 228us) server_negotiation.cc:407] Connection header received
0812 21:03:59.142829 (+ 482us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:03:59.142830 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:03:59.142841 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:03:59.142891 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:03:59.143506 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:59.144013 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:59.145090 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:59.145283 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:59.145337 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:59.147953 (+ 2616us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:03:59.147955 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:03:59.147955 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:03:59.147963 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:03:59.149471 (+ 1508us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:59.149875 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:59.149877 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:59.149877 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:59.149979 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:59.150229 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:59.150230 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:59.150230 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:59.150393 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:03:59.150455 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0812 21:03:59.150743 (+ 288us) server_negotiation.cc:299] Negotiation successful
0812 21:03:59.150832 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260812 21:03:59.151152 24128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:59.142032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:43007 (local address 127.0.0.1:38712)
0812 21:03:59.142219 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0812 21:03:59.142257 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 21:03:59.142747 (+ 490us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:03:59.142954 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:03:59.142957 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:03:59.143234 (+ 277us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:03:59.143437 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:59.143441 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:59.144115 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:59.144117 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:59.145011 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:59.145013 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:59.145396 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:59.145397 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:59.145562 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:59.146196 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:03:59.146207 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:03:59.147822 (+ 1615us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:03:59.149551 (+ 1729us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:03:59.149556 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:03:59.149557 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:03:59.149796 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:03:59.150042 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:03:59.150043 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:03:59.150043 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:03:59.150155 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:03:59.150455 (+ 300us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:03:59.150461 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:03:59.150596 (+ 135us) client_negotiation.cc:769] Sending connection context
0812 21:03:59.150708 (+ 112us) client_negotiation.cc:240] Negotiation successful
0812 21:03:59.150867 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":106,"threads_started":1}
Aug 12 21:03:59 dist-test-slave-vpvm krb5kdc[23147](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568638, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.11.254.189@KRBTEST.COM
I20260812 21:03:59.151883 24126 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.254.189" port: 36155 } has no permanent_uuid. Determining permanent_uuid...
I20260812 21:03:59.154707 24134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:59.146025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41242 (local address 127.11.254.189:36155)
0812 21:03:59.146230 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0812 21:03:59.146235 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 21:03:59.146260 (+ 25us) server_negotiation.cc:407] Connection header received
0812 21:03:59.146281 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:03:59.146283 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:03:59.146301 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:03:59.146364 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:03:59.146998 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:59.147979 (+ 981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:59.149032 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:59.149391 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:59.149448 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:59.151831 (+ 2383us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:03:59.151836 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:03:59.151836 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:03:59.151843 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:03:59.153587 (+ 1744us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:59.153902 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:59.153905 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:59.153905 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:59.153988 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:59.154143 (+ 155us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:59.154144 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:59.154144 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:59.154284 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:03:59.154403 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0812 21:03:59.154488 (+ 85us) server_negotiation.cc:299] Negotiation successful
0812 21:03:59.154592 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":171,"thread_start_us":133,"threads_started":1}
Aug 12 21:03:59 dist-test-slave-vpvm krb5kdc[23147](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568639, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.254.189@KRBTEST.COM for oryx/127.11.254.189@KRBTEST.COM
I20260812 21:03:59.158718 24128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:59.152136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.189:36155 (local address 127.0.0.1:41244)
0812 21:03:59.152325 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0812 21:03:59.152334 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 21:03:59.152367 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:03:59.152490 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:03:59.152491 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:03:59.152649 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:03:59.152703 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:59.152705 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:59.153431 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:59.153432 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:59.154099 (+ 667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:59.154101 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:59.154475 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:59.154476 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:59.154585 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:59.154903 (+ 318us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:03:59.154908 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:03:59.156039 (+ 1131us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:03:59.157784 (+ 1745us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:03:59.157785 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:03:59.157785 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:03:59.157927 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:03:59.158185 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:03:59.158186 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:03:59.158186 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:03:59.158248 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:03:59.158437 (+ 189us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:03:59.158437 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:03:59.158470 (+ 33us) client_negotiation.cc:769] Sending connection context
0812 21:03:59.158566 (+ 96us) client_negotiation.cc:240] Negotiation successful
0812 21:03:59.158650 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176,"mutex_wait_us":35}
I20260812 21:03:59.158823 24137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:59.152209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41244 (local address 127.11.254.189:36155)
0812 21:03:59.152351 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0812 21:03:59.152353 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:03:59.152360 (+ 7us) server_negotiation.cc:407] Connection header received
0812 21:03:59.152403 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:03:59.152403 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:03:59.152411 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:03:59.152464 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:03:59.152766 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:59.153355 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:59.154175 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:59.154405 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:59.154470 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:59.156128 (+ 1658us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:03:59.156129 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:03:59.156129 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:03:59.156135 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:03:59.157719 (+ 1584us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:59.158020 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:59.158021 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:59.158021 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:59.158097 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:59.158307 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:59.158307 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:59.158307 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:59.158399 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:03:59.158436 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0812 21:03:59.158587 (+ 151us) server_negotiation.cc:299] Negotiation successful
0812 21:03:59.158696 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":50,"server-negotiator.queue_time_us":123,"thread_start_us":86,"threads_started":1}
I20260812 21:03:59.160079 24126 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5026b3627f2e49f8b4e7cf1521420cd1: Bootstrap starting.
I20260812 21:03:59.160714 24126 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5026b3627f2e49f8b4e7cf1521420cd1: Neither blocks nor log segments found. Creating new log.
I20260812 21:03:59.160959 24126 log.cc:824] T 00000000000000000000000000000000 P 5026b3627f2e49f8b4e7cf1521420cd1: Log is configured to *not* fsync() on all Append() calls
I20260812 21:03:59.161789 24126 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5026b3627f2e49f8b4e7cf1521420cd1: No bootstrap required, opened a new log
I20260812 21:03:59.163547 24126 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5026b3627f2e49f8b4e7cf1521420cd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c90e0f61a5354abbbf96692370a65ebd" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 43007 } } peers { permanent_uuid: "5026b3627f2e49f8b4e7cf1521420cd1" member_type: VOTER last_known_addr { host: "127.11.254.189" port: 36155 } }
I20260812 21:03:59.163692 24126 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5026b3627f2e49f8b4e7cf1521420cd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:03:59.163726 24126 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5026b3627f2e49f8b4e7cf1521420cd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5026b3627f2e49f8b4e7cf1521420cd1, State: Initialized, Role: FOLLOWER
I20260812 21:03:59.163831 24126 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5026b3627f2e49f8b4e7cf1521420cd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c90e0f61a5354abbbf96692370a65ebd" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 43007 } } peers { permanent_uuid: "5026b3627f2e49f8b4e7cf1521420cd1" member_type: VOTER last_known_addr { host: "127.11.254.189" port: 36155 } }
I20260812 21:03:59.164182 24126 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5026b3627f2e49f8b4e7cf1521420cd1 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 21:03:59.164160 24140 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5026b3627f2e49f8b4e7cf1521420cd1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c90e0f61a5354abbbf96692370a65ebd" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 43007 } } peers { permanent_uuid: "5026b3627f2e49f8b4e7cf1521420cd1" member_type: VOTER last_known_addr { host: "127.11.254.189" port: 36155 } } }
I20260812 21:03:59.164259 24140 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5026b3627f2e49f8b4e7cf1521420cd1 [sys.catalog]: This master's current role is: FOLLOWER
W20260812 21:03:59.166132 24158 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5026b3627f2e49f8b4e7cf1521420cd1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 21:03:59.166198 24158 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 21:03:59.167950 24025 master_runner.cc:428] Detected that this master 5026b3627f2e49f8b4e7cf1521420cd1 is joining an existing cluster
I20260812 21:03:59.168028 24025 master_runner.cc:432] Initiating AddMaster RPC to add 127.11.254.189:36155
I20260812 21:03:59.176194 24172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:59.169261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.189:36155 (local address 127.0.0.1:41246)
0812 21:03:59.169369 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0812 21:03:59.169377 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0812 21:03:59.169435 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:03:59.169570 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:03:59.169570 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:03:59.169808 (+ 238us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:03:59.169885 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:59.169888 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:59.170863 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:59.170864 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:59.171784 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:59.171786 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:59.172316 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:59.172317 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:59.172372 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:59.173116 (+ 744us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:03:59.173121 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:03:59.173575 (+ 454us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:03:59.175193 (+ 1618us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:03:59.175194 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:03:59.175194 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:03:59.175422 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:03:59.175591 (+ 169us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:03:59.175591 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:03:59.175592 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:03:59.175642 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:03:59.175848 (+ 206us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:03:59.175849 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:03:59.175887 (+ 38us) client_negotiation.cc:769] Sending connection context
0812 21:03:59.175946 (+ 59us) client_negotiation.cc:240] Negotiation successful
0812 21:03:59.176051 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":96,"mutex_wait_us":26,"thread_start_us":64,"threads_started":1}
I20260812 21:03:59.176200 24137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:59.169289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41246 (local address 127.11.254.189:36155)
0812 21:03:59.169326 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0812 21:03:59.169328 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:03:59.169439 (+ 111us) server_negotiation.cc:407] Connection header received
0812 21:03:59.169480 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:03:59.169481 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:03:59.169492 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:03:59.169563 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:03:59.170011 (+ 448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:59.170798 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:59.171871 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:59.172185 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:59.172254 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:59.173695 (+ 1441us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:03:59.173696 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:03:59.173696 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:03:59.173703 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:03:59.175086 (+ 1383us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:59.175492 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:59.175493 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:59.175494 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:59.175541 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:59.175691 (+ 150us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:59.175691 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:59.175692 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:59.175797 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:03:59.175852 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0812 21:03:59.175969 (+ 117us) server_negotiation.cc:299] Negotiation successful
0812 21:03:59.176116 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":80,"server-negotiator.queue_time_us":21}
I20260812 21:03:59.176344 23283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:59.169150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38720 (local address 127.11.254.190:43007)
0812 21:03:59.169200 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0812 21:03:59.169202 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:03:59.169265 (+ 63us) server_negotiation.cc:407] Connection header received
0812 21:03:59.169396 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:03:59.169397 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:03:59.169411 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:03:59.169463 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:03:59.170253 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:59.170757 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:59.171930 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:59.172129 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:59.172181 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:59.173421 (+ 1240us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:03:59.173423 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:03:59.173423 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:03:59.173431 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:03:59.175054 (+ 1623us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:59.175461 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:59.175462 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:59.175463 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:59.175526 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:59.175768 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:59.175769 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:59.175769 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:59.175908 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:03:59.175953 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0812 21:03:59.176154 (+ 201us) server_negotiation.cc:299] Negotiation successful
0812 21:03:59.176237 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260812 21:03:59.176904 24170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:59.169034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:43007 (local address 127.0.0.1:38720)
0812 21:03:59.169194 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0812 21:03:59.169206 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 21:03:59.169304 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:03:59.169514 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:03:59.169514 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:03:59.169854 (+ 340us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:03:59.170174 (+ 320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:59.170176 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:59.170829 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:59.170830 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:59.171864 (+ 1034us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:59.171866 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:59.172333 (+ 467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:59.172334 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:59.172381 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:59.172906 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:03:59.172911 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:03:59.173343 (+ 432us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:03:59.175118 (+ 1775us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:03:59.175119 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:03:59.175120 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:03:59.175392 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:03:59.175615 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:03:59.175616 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:03:59.175616 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:03:59.175693 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:03:59.176000 (+ 307us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:03:59.176001 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:03:59.176043 (+ 42us) client_negotiation.cc:769] Sending connection context
0812 21:03:59.176695 (+ 652us) client_negotiation.cc:240] Negotiation successful
0812 21:03:59.176804 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":89,"threads_started":1}
Aug 12 21:03:59 dist-test-slave-vpvm krb5kdc[23147](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568638, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.254.190@KRBTEST.COM for oryx/127.11.254.189@KRBTEST.COM
I20260812 21:03:59.186017 24137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:59.179112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41254 (local address 127.11.254.189:36155)
0812 21:03:59.179160 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0812 21:03:59.179161 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 21:03:59.179316 (+ 155us) server_negotiation.cc:407] Connection header received
0812 21:03:59.179392 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:03:59.179393 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:03:59.179403 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:03:59.179447 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:03:59.179858 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:59.180310 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:59.181066 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:59.181263 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:59.181317 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:59.183456 (+ 2139us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:03:59.183457 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:03:59.183458 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:03:59.183465 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:03:59.184719 (+ 1254us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:59.185207 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:59.185208 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:59.185208 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:59.185254 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:59.185491 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:59.185492 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:59.185492 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:59.185593 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:03:59.185665 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0812 21:03:59.185847 (+ 182us) server_negotiation.cc:299] Negotiation successful
0812 21:03:59.185907 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260812 21:03:59.186142 24173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:59.179068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.189:36155 (local address 127.0.0.1:41254)
0812 21:03:59.179223 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0812 21:03:59.179258 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0812 21:03:59.179329 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:03:59.179451 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:03:59.179454 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:03:59.179732 (+ 278us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:03:59.179784 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:59.179786 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:59.180389 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:59.180390 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:59.180990 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:59.180992 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:59.181323 (+ 331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:59.181324 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:59.181410 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:59.181828 (+ 418us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:03:59.181833 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:03:59.183368 (+ 1535us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:03:59.184824 (+ 1456us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:03:59.184834 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:03:59.184836 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:03:59.185134 (+ 298us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:03:59.185307 (+ 173us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:03:59.185308 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:03:59.185308 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:03:59.185378 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:03:59.185664 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:03:59.185672 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:03:59.185735 (+ 63us) client_negotiation.cc:769] Sending connection context
0812 21:03:59.185830 (+ 95us) client_negotiation.cc:240] Negotiation successful
0812 21:03:59.186017 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":61,"threads_started":1}
I20260812 21:03:59.186859 23224 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.11.254.189:36155: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "5026b3627f2e49f8b4e7cf1521420cd1"
member_type: NON_VOTER
last_known_addr {
host: "127.11.254.189"
port: 36155
}
attrs {
promote: true
}
}
dest_uuid: "c90e0f61a5354abbbf96692370a65ebd"
cas_config_opid_index: -1
I20260812 21:03:59.187327 23224 consensus_queue.cc:237] T 00000000000000000000000000000000 P c90e0f61a5354abbbf96692370a65ebd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c90e0f61a5354abbbf96692370a65ebd" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 43007 } } peers { permanent_uuid: "5026b3627f2e49f8b4e7cf1521420cd1" member_type: NON_VOTER last_known_addr { host: "127.11.254.189" port: 36155 } attrs { promote: true } }
I20260812 21:03:59.188001 24175 raft_consensus.cc:2932] T 00000000000000000000000000000000 P c90e0f61a5354abbbf96692370a65ebd [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 5026b3627f2e49f8b4e7cf1521420cd1 (127.11.254.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c90e0f61a5354abbbf96692370a65ebd" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 43007 } } peers { permanent_uuid: "5026b3627f2e49f8b4e7cf1521420cd1" member_type: NON_VOTER last_known_addr { host: "127.11.254.189" port: 36155 } attrs { promote: true } } }
I20260812 21:03:59.188262 24086 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5026b3627f2e49f8b4e7cf1521420cd1 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:03:59.189062 24086 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5026b3627f2e49f8b4e7cf1521420cd1 [term 1 FOLLOWER]: Refusing update from remote peer c90e0f61a5354abbbf96692370a65ebd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260812 21:03:59.189138 24175 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.11.254.189:36155
I20260812 21:03:59.189231 24175 sys_catalog.cc:455] T 00000000000000000000000000000000 P c90e0f61a5354abbbf96692370a65ebd [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "c90e0f61a5354abbbf96692370a65ebd" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c90e0f61a5354abbbf96692370a65ebd" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 43007 } } peers { permanent_uuid: "5026b3627f2e49f8b4e7cf1521420cd1" member_type: NON_VOTER last_known_addr { host: "127.11.254.189" port: 36155 } attrs { promote: true } } }
I20260812 21:03:59.189289 24175 sys_catalog.cc:458] T 00000000000000000000000000000000 P c90e0f61a5354abbbf96692370a65ebd [sys.catalog]: This master's current role is: LEADER
I20260812 21:03:59.189527 24175 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c90e0f61a5354abbbf96692370a65ebd [LEADER]: Connected to new peer: Peer: permanent_uuid: "5026b3627f2e49f8b4e7cf1521420cd1" member_type: NON_VOTER last_known_addr { host: "127.11.254.189" port: 36155 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 21:03:59.189574 24025 master.cc:537] Master@127.11.254.189:36155 shutting down...
W20260812 21:03:59.190379 23197 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
I20260812 21:03:59.192288 24025 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 5026b3627f2e49f8b4e7cf1521420cd1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 21:03:59.192387 24025 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 5026b3627f2e49f8b4e7cf1521420cd1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 21:03:59.192441 24025 tablet_replica.cc:330] T 00000000000000000000000000000000 P 5026b3627f2e49f8b4e7cf1521420cd1: stopping tablet replica
I20260812 21:03:59.198441 24173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:59.198287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.189:36155 (local address 127.0.0.1:41264)
0812 21:03:59.198345 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0812 21:03:59.198394 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.189:36155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260812 21:03:59.198568 23197 consensus_peers.cc:487] T 00000000000000000000000000000000 P c90e0f61a5354abbbf96692370a65ebd -> Peer 5026b3627f2e49f8b4e7cf1521420cd1 (127.11.254.189:36155): Couldn't send request to peer 5026b3627f2e49f8b4e7cf1521420cd1. Status: Network error: Client connection negotiation failed: client connection to 127.11.254.189:36155: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260812 21:03:59.205768 24025 master.cc:552] Master@127.11.254.189:36155 shutdown complete.
I20260812 21:03:59.205835 24025 master_runner.cc:305] Clearing existing system tablet
I20260812 21:03:59.206012 24025 fs_manager.cc:359] Metadata directory not provided
I20260812 21:03:59.206065 24025 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:03:59.206790 24025 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.001s
I20260812 21:03:59.207337 24181 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:03:59.207501 24025 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:03:59.207567 24025 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 21:03:59.207592 24025 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/master-1/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/master-1/wal
uuid: "5026b3627f2e49f8b4e7cf1521420cd1"
format_stamp: "Formatted at 2026-08-12 21:03:59 on dist-test-slave-vpvm"
server_key: "6ec19b8e883d1b3ea2e279d8f97038c7"
server_key_iv: "ee27801f1e0dc85f289f2b3d5ff6acdb"
server_key_version: "encryptionkey@0"
I20260812 21:03:59.208603 24025 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 5026b3627f2e49f8b4e7cf1521420cd1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 21:03:59.210219 24025 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 5026b3627f2e49f8b4e7cf1521420cd1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260812 21:03:59.210305 24025 log.cc:1196] T 00000000000000000000000000000000 P 5026b3627f2e49f8b4e7cf1521420cd1: Deleting WAL directory at /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260812 21:03:59.210521 24025 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 5026b3627f2e49f8b4e7cf1521420cd1: Deleting consensus metadata
I20260812 21:03:59.210708 24025 master_runner.cc:315] Copying system tablet from 127.11.254.190:43007
I20260812 21:03:59.211380 24025 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 5026b3627f2e49f8b4e7cf1521420cd1: tablet copy: Beginning tablet copy session from remote peer at address 127.11.254.190:43007
I20260812 21:03:59.217733 23283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:59.211797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38732 (local address 127.11.254.190:43007)
0812 21:03:59.211848 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0812 21:03:59.211850 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:03:59.211909 (+ 59us) server_negotiation.cc:407] Connection header received
0812 21:03:59.211998 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:03:59.211999 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:03:59.212009 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:03:59.212046 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:03:59.212422 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:59.212937 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:59.214003 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:59.214214 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:59.214269 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:59.215344 (+ 1075us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:03:59.215345 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:03:59.215346 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:03:59.215352 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:03:59.216616 (+ 1264us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:59.217006 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:59.217007 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:59.217007 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:59.217055 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:59.217227 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:59.217228 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:59.217228 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:59.217335 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:03:59.217383 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0812 21:03:59.217578 (+ 195us) server_negotiation.cc:299] Negotiation successful
0812 21:03:59.217662 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260812 21:03:59.217787 24192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:59.211699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:43007 (local address 127.0.0.1:38732)
0812 21:03:59.211827 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0812 21:03:59.211840 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 21:03:59.211928 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:03:59.212056 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:03:59.212057 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:03:59.212273 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:03:59.212355 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:59.212358 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:59.213001 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:59.213002 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:59.213865 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:59.213868 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:59.214288 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:59.214289 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:59.214374 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:59.214803 (+ 429us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:03:59.214809 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:03:59.215250 (+ 441us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:03:59.216704 (+ 1454us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:03:59.216705 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:03:59.216706 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:03:59.216942 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:03:59.217110 (+ 168us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:03:59.217110 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:03:59.217111 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:03:59.217176 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:03:59.217388 (+ 212us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:03:59.217412 (+ 24us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:03:59.217478 (+ 66us) client_negotiation.cc:769] Sending connection context
0812 21:03:59.217551 (+ 73us) client_negotiation.cc:240] Negotiation successful
0812 21:03:59.217669 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":80,"threads_started":1}
I20260812 21:03:59.218189 23257 tablet_copy_service.cc:140] P c90e0f61a5354abbbf96692370a65ebd: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 5026b3627f2e49f8b4e7cf1521420cd1 ({username='oryx', principal='oryx/127.11.254.189@KRBTEST.COM'} at 127.0.0.1:38732)
I20260812 21:03:59.218319 23257 tablet_copy_service.cc:161] P c90e0f61a5354abbbf96692370a65ebd: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 5026b3627f2e49f8b4e7cf1521420cd1 at {username='oryx', principal='oryx/127.11.254.189@KRBTEST.COM'} at 127.0.0.1:38732: session id = 5026b3627f2e49f8b4e7cf1521420cd1-00000000000000000000000000000000
I20260812 21:03:59.219447 23257 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P c90e0f61a5354abbbf96692370a65ebd: Tablet Copy: opened 0 blocks and 1 log segments
I20260812 21:03:59.220184 24025 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260812 21:03:59.221346 24025 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 5026b3627f2e49f8b4e7cf1521420cd1: tablet copy: Starting download of 0 data blocks...
I20260812 21:03:59.221498 24025 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 5026b3627f2e49f8b4e7cf1521420cd1: tablet copy: Starting download of 1 WAL segments...
I20260812 21:03:59.222574 24025 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 5026b3627f2e49f8b4e7cf1521420cd1: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260812 21:03:59.223419 23257 tablet_copy_service.cc:342] P c90e0f61a5354abbbf96692370a65ebd: Request end of tablet copy session 5026b3627f2e49f8b4e7cf1521420cd1-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.11.254.189@KRBTEST.COM'} at 127.0.0.1:38732
I20260812 21:03:59.223474 23257 tablet_copy_service.cc:433] P c90e0f61a5354abbbf96692370a65ebd: ending tablet copy session 5026b3627f2e49f8b4e7cf1521420cd1-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 5026b3627f2e49f8b4e7cf1521420cd1
I20260812 21:03:59.226042 24025 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:03:59.227238 24193 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:03:59.227339 24198 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:03:59.227334 24196 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:03:59.227587 24025 server_base.cc:1034] running on GCE node
I20260812 21:03:59.227715 24025 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:03:59.227859 24025 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:03:59.228981 24025 hybrid_clock.cc:648] HybridClock initialized: now 1786568639228974 us; error 24 us; skew 500 ppm
Aug 12 21:03:59 dist-test-slave-vpvm krb5kdc[23147](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568639, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.254.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:03:59.230315 24025 init.cc:377] Logged in from keytab as oryx/127.11.254.189@KRBTEST.COM (short username oryx)
I20260812 21:03:59.230579 24025 webserver.cc:460] Webserver started at http://127.11.254.189:40893/ using document root <none> and password file <none>
I20260812 21:03:59.230700 24025 fs_manager.cc:359] Metadata directory not provided
I20260812 21:03:59.230732 24025 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:03:59.231323 24025 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260812 21:03:59.231856 24216 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:03:59.232004 24025 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 21:03:59.232079 24025 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/master-1/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/master-1/wal
uuid: "5026b3627f2e49f8b4e7cf1521420cd1"
format_stamp: "Formatted at 2026-08-12 21:03:59 on dist-test-slave-vpvm"
server_key: "6ec19b8e883d1b3ea2e279d8f97038c7"
server_key_iv: "ee27801f1e0dc85f289f2b3d5ff6acdb"
server_key_version: "encryptionkey@0"
I20260812 21:03:59.232144 24025 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:03:59.253741 24025 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:03:59.255316 24025 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:03:59.258378 24025 rpc_server.cc:307] RPC server started. Bound to: 127.11.254.189:36155
I20260812 21:03:59.258396 24307 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.254.189:36155 every 8 connection(s)
I20260812 21:03:59.258709 24025 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/master-1/data/info.pb
I20260812 21:03:59.259714 24308 sys_catalog.cc:263] Verifying existing consensus state
I20260812 21:03:59.260304 24308 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5026b3627f2e49f8b4e7cf1521420cd1: Bootstrap starting.
I20260812 21:03:59.263473 24308 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5026b3627f2e49f8b4e7cf1521420cd1: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 21:03:59.263727 24308 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5026b3627f2e49f8b4e7cf1521420cd1: Bootstrap complete.
I20260812 21:03:59.263929 24308 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5026b3627f2e49f8b4e7cf1521420cd1 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c90e0f61a5354abbbf96692370a65ebd" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 43007 } } peers { permanent_uuid: "5026b3627f2e49f8b4e7cf1521420cd1" member_type: NON_VOTER last_known_addr { host: "127.11.254.189" port: 36155 } attrs { promote: true } }
I20260812 21:03:59.264020 24308 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5026b3627f2e49f8b4e7cf1521420cd1 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 5026b3627f2e49f8b4e7cf1521420cd1, State: Initialized, Role: LEARNER
I20260812 21:03:59.264065 24308 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5026b3627f2e49f8b4e7cf1521420cd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c90e0f61a5354abbbf96692370a65ebd" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 43007 } } peers { permanent_uuid: "5026b3627f2e49f8b4e7cf1521420cd1" member_type: NON_VOTER last_known_addr { host: "127.11.254.189" port: 36155 } attrs { promote: true } }
I20260812 21:03:59.264279 24308 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5026b3627f2e49f8b4e7cf1521420cd1 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 21:03:59.264245 24316 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5026b3627f2e49f8b4e7cf1521420cd1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c90e0f61a5354abbbf96692370a65ebd" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 43007 } } peers { permanent_uuid: "5026b3627f2e49f8b4e7cf1521420cd1" member_type: NON_VOTER last_known_addr { host: "127.11.254.189" port: 36155 } attrs { promote: true } } }
I20260812 21:03:59.264338 24316 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5026b3627f2e49f8b4e7cf1521420cd1 [sys.catalog]: This master's current role is: LEARNER
I20260812 21:03:59.265769 24344 catalog_manager.cc:1259] Loaded cluster ID: b39e89298b2f46b5a2e07cc9a4507645
I20260812 21:03:59.265818 24344 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 5026b3627f2e49f8b4e7cf1521420cd1: loading cluster ID for follower catalog manager: success
I20260812 21:03:59.266700 24344 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 5026b3627f2e49f8b4e7cf1521420cd1: acquiring CA information for follower catalog manager: success
I20260812 21:03:59.266965 24344 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 5026b3627f2e49f8b4e7cf1521420cd1: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 21:03:59.313756 24345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:59.306872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41280 (local address 127.11.254.189:36155)
0812 21:03:59.307027 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0812 21:03:59.307030 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 21:03:59.307039 (+ 9us) server_negotiation.cc:407] Connection header received
0812 21:03:59.307064 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:03:59.307065 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:03:59.307074 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:03:59.307136 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:03:59.307851 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:59.308581 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:59.309660 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:59.309860 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:59.309922 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:59.311390 (+ 1468us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:03:59.311392 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:03:59.311392 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:03:59.311399 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:03:59.312758 (+ 1359us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:59.313123 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:59.313124 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:59.313124 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:59.313173 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:59.313339 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:59.313339 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:59.313339 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:59.313435 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:03:59.313501 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0812 21:03:59.313619 (+ 118us) server_negotiation.cc:299] Negotiation successful
0812 21:03:59.313682 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":85,"threads_started":1}
WARNING: no policy specified for oryx/127.11.254.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.254.188@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.254.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/krb5kdc/oryx_127.11.254.188.keytab.
Entry for principal oryx/127.11.254.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/krb5kdc/oryx_127.11.254.188.keytab.
I20260812 21:03:59.341646 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.254.188:36857
--webserver_interface=127.11.254.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/krb5kdc/oryx_127.11.254.188.keytab
--principal=oryx/127.11.254.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.254.148:45807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.254.190:43007,127.11.254.189:36155,127.11.254.188:36857
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:03:59.424626 24353 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:03:59.424842 24353 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:03:59.424873 24353 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:03:59.426079 24353 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 21:03:59.426121 24353 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:03:59.426139 24353 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:03:59.426154 24353 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 21:03:59.426172 24353 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 21:03:59.427703 24353 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.254.148:45807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.11.254.190:43007,127.11.254.189:36155,127.11.254.188:36857
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/krb5kdc/oryx_127.11.254.188.keytab
--principal=oryx/127.11.254.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.254.188:36857
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.11.254.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24353
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:03:59.427953 24353 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:03:59.428167 24353 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:03:59.433008 24359 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:03:59.433001 24360 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:03:59.433001 24364 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:03:59.433270 24353 server_base.cc:1034] running on GCE node
I20260812 21:03:59.433542 24353 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:03:59.433845 24353 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:03:59.435003 24353 hybrid_clock.cc:648] HybridClock initialized: now 1786568639434988 us; error 31 us; skew 500 ppm
Aug 12 21:03:59 dist-test-slave-vpvm krb5kdc[23147](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568639, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.254.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:03:59.437599 24353 init.cc:377] Logged in from keytab as oryx/127.11.254.188@KRBTEST.COM (short username oryx)
I20260812 21:03:59.438119 24353 webserver.cc:460] Webserver started at http://127.11.254.188:33283/ using document root <none> and password file <none>
I20260812 21:03:59.438335 24353 fs_manager.cc:359] Metadata directory not provided
I20260812 21:03:59.438375 24353 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:03:59.438491 24353 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:03:59.439540 24353 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/master-2/data/instance:
uuid: "7e4700f28485420fbba29f2e545c71df"
format_stamp: "Formatted at 2026-08-12 21:03:59 on dist-test-slave-vpvm"
server_key: "9945d4ff1f12643293ae9659102aa78e"
server_key_iv: "775a9a0e271a356a1150bab9644e76f5"
server_key_version: "encryptionkey@0"
I20260812 21:03:59.439849 24353 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/master-2/wal/instance:
uuid: "7e4700f28485420fbba29f2e545c71df"
format_stamp: "Formatted at 2026-08-12 21:03:59 on dist-test-slave-vpvm"
server_key: "9945d4ff1f12643293ae9659102aa78e"
server_key_iv: "775a9a0e271a356a1150bab9644e76f5"
server_key_version: "encryptionkey@0"
I20260812 21:03:59.441216 24353 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 21:03:59.441974 24377 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:03:59.442140 24353 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 21:03:59.442205 24353 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/master-2/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/master-2/wal
uuid: "7e4700f28485420fbba29f2e545c71df"
format_stamp: "Formatted at 2026-08-12 21:03:59 on dist-test-slave-vpvm"
server_key: "9945d4ff1f12643293ae9659102aa78e"
server_key_iv: "775a9a0e271a356a1150bab9644e76f5"
server_key_version: "encryptionkey@0"
I20260812 21:03:59.442276 24353 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:03:59.451081 24353 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:03:59.453135 24353 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:03:59.453277 24353 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:03:59.457161 24353 rpc_server.cc:307] RPC server started. Bound to: 127.11.254.188:36857
I20260812 21:03:59.457202 24459 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.254.188:36857 every 8 connection(s)
I20260812 21:03:59.457538 24353 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/master-2/data/info.pb
I20260812 21:03:59.458603 24462 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:03:59.460812 24462 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.254.190" port: 43007 } has no permanent_uuid. Determining permanent_uuid...
I20260812 21:03:59.467350 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 24353
Aug 12 21:03:59 dist-test-slave-vpvm krb5kdc[23147](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568639, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.254.188@KRBTEST.COM for oryx/127.11.254.190@KRBTEST.COM
I20260812 21:03:59.471289 23283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:59.461729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38736 (local address 127.11.254.190:43007)
0812 21:03:59.461789 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0812 21:03:59.461791 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:03:59.462154 (+ 363us) server_negotiation.cc:407] Connection header received
0812 21:03:59.462910 (+ 756us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:03:59.462911 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:03:59.462922 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:03:59.462991 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:03:59.463808 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:59.464376 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:59.465658 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:59.465904 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:59.465995 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:59.468550 (+ 2555us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:03:59.468553 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:03:59.468553 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:03:59.468564 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:03:59.469882 (+ 1318us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:59.470298 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:59.470299 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:59.470300 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:59.470357 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:59.470648 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:59.470649 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:59.470649 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:59.470789 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:03:59.470859 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 21:03:59.471103 (+ 244us) server_negotiation.cc:299] Negotiation successful
0812 21:03:59.471195 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260812 21:03:59.471589 24465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:59.461599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:43007 (local address 127.0.0.1:38736)
0812 21:03:59.461937 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0812 21:03:59.462056 (+ 119us) client_negotiation.cc:174] Beginning negotiation
0812 21:03:59.462817 (+ 761us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:03:59.463045 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:03:59.463048 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:03:59.463399 (+ 351us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:03:59.463733 (+ 334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:59.463738 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:59.464445 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:59.464446 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:59.465580 (+ 1134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:59.465583 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:59.466097 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:59.466099 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:59.466210 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:59.466704 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:03:59.466713 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:03:59.468391 (+ 1678us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:03:59.469996 (+ 1605us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:03:59.470002 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:03:59.470005 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:03:59.470228 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:03:59.470424 (+ 196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:03:59.470425 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:03:59.470425 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:03:59.470572 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:03:59.470851 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:03:59.470855 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:03:59.470978 (+ 123us) client_negotiation.cc:769] Sending connection context
0812 21:03:59.471058 (+ 80us) client_negotiation.cc:240] Negotiation successful
0812 21:03:59.471194 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":224,"threads_started":1}
I20260812 21:03:59.472174 24462 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.254.189" port: 36155 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 21:03:59 dist-test-slave-vpvm krb5kdc[23147](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568638, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.11.254.188@KRBTEST.COM
Aug 12 21:03:59 dist-test-slave-vpvm krb5kdc[23147](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568639, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.254.188@KRBTEST.COM for oryx/127.11.254.189@KRBTEST.COM
I20260812 21:03:59.476727 24469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:59.467944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32950 (local address 127.11.254.188:36857)
0812 21:03:59.468213 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0812 21:03:59.468217 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 21:03:59.468233 (+ 16us) server_negotiation.cc:407] Connection header received
0812 21:03:59.468251 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:03:59.468253 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:03:59.468289 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:03:59.468361 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:03:59.469088 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:59.469832 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:59.470653 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:59.471156 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:59.471225 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:59.473761 (+ 2536us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:03:59.473768 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:03:59.473768 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:03:59.473774 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:03:59.475632 (+ 1858us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:59.475950 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:59.475953 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:59.475954 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:59.476009 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:59.476174 (+ 165us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:59.476174 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:59.476175 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:59.476331 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:03:59.476470 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0812 21:03:59.476537 (+ 67us) server_negotiation.cc:299] Negotiation successful
0812 21:03:59.476607 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":217,"thread_start_us":99,"threads_started":1}
I20260812 21:03:59.478686 24465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:59.472410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.189:36155 (local address 127.0.0.1:41296)
0812 21:03:59.472521 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0812 21:03:59.472529 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0812 21:03:59.472560 (+ 31us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:03:59.472687 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:03:59.472688 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:03:59.472826 (+ 138us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:03:59.472975 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:59.472977 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:59.474039 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:59.474040 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:59.474529 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:59.474531 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:59.474942 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:59.474943 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:59.475012 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:59.475321 (+ 309us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:03:59.475324 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:03:59.476503 (+ 1179us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:03:59.477934 (+ 1431us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:03:59.477935 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:03:59.477935 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:03:59.478116 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:03:59.478265 (+ 149us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:03:59.478266 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:03:59.478266 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:03:59.478307 (+ 41us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:03:59.478490 (+ 183us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:03:59.478490 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:03:59.478522 (+ 32us) client_negotiation.cc:769] Sending connection context
0812 21:03:59.478566 (+ 44us) client_negotiation.cc:240] Negotiation successful
0812 21:03:59.478630 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25}
I20260812 21:03:59.478708 24345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:59.472489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41296 (local address 127.11.254.189:36155)
0812 21:03:59.472536 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0812 21:03:59.472538 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:03:59.472549 (+ 11us) server_negotiation.cc:407] Connection header received
0812 21:03:59.472608 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:03:59.472609 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:03:59.472624 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:03:59.472674 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:03:59.473042 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:59.473678 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:59.474608 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:59.474880 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:59.474934 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:59.476585 (+ 1651us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:03:59.476586 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:03:59.476586 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:03:59.476594 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:03:59.477863 (+ 1269us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:59.478175 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:59.478176 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:59.478176 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:59.478222 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:59.478337 (+ 115us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:59.478337 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:59.478337 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:59.478421 (+ 84us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:03:59.478462 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0812 21:03:59.478583 (+ 121us) server_negotiation.cc:299] Negotiation successful
0812 21:03:59.478648 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 21:03:59.479087 24462 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.254.188" port: 36857 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 21:03:59 dist-test-slave-vpvm krb5kdc[23147](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568639, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.254.188@KRBTEST.COM for oryx/127.11.254.188@KRBTEST.COM
I20260812 21:03:59.484501 24465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:59.479311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.188:36857 (local address 127.0.0.1:32962)
0812 21:03:59.479352 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0812 21:03:59.479362 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 21:03:59.479404 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:03:59.479492 (+ 88us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:03:59.479493 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:03:59.479594 (+ 101us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:03:59.479621 (+ 27us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:59.479622 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:59.480285 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:59.480286 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:59.480753 (+ 467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:59.480754 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:59.481083 (+ 329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:59.481084 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:59.481130 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:59.481424 (+ 294us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:03:59.481427 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:03:59.482481 (+ 1054us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:03:59.483794 (+ 1313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:03:59.483794 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:03:59.483795 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:03:59.483930 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:03:59.484058 (+ 128us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:03:59.484058 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:03:59.484058 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:03:59.484098 (+ 40us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:03:59.484267 (+ 169us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:03:59.484267 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:03:59.484299 (+ 32us) client_negotiation.cc:769] Sending connection context
0812 21:03:59.484342 (+ 43us) client_negotiation.cc:240] Negotiation successful
0812 21:03:59.484412 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25}
I20260812 21:03:59.484550 24469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:59.479342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32962 (local address 127.11.254.188:36857)
0812 21:03:59.479370 (+ 28us) server_negotiation.cc:206] Beginning negotiation
0812 21:03:59.479372 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:03:59.479391 (+ 19us) server_negotiation.cc:407] Connection header received
0812 21:03:59.479441 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:03:59.479441 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:03:59.479449 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:03:59.479490 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:03:59.479665 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:59.480197 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:59.480823 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:59.481035 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:59.481080 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:59.482553 (+ 1473us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:03:59.482554 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:03:59.482555 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:03:59.482561 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:03:59.483734 (+ 1173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:59.483979 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:59.483980 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:59.483980 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:59.484019 (+ 39us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:59.484148 (+ 129us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:59.484148 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:59.484148 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:59.484230 (+ 82us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:03:59.484269 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0812 21:03:59.484358 (+ 89us) server_negotiation.cc:299] Negotiation successful
0812 21:03:59.484453 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":40,"server-negotiator.queue_time_us":14}
I20260812 21:03:59.485925 24462 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e4700f28485420fbba29f2e545c71df: Bootstrap starting.
I20260812 21:03:59.487299 24462 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7e4700f28485420fbba29f2e545c71df: Neither blocks nor log segments found. Creating new log.
I20260812 21:03:59.487906 24462 log.cc:824] T 00000000000000000000000000000000 P 7e4700f28485420fbba29f2e545c71df: Log is configured to *not* fsync() on all Append() calls
I20260812 21:03:59.488718 24462 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e4700f28485420fbba29f2e545c71df: No bootstrap required, opened a new log
I20260812 21:03:59.491855 24462 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7e4700f28485420fbba29f2e545c71df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c90e0f61a5354abbbf96692370a65ebd" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 43007 } } peers { permanent_uuid: "5026b3627f2e49f8b4e7cf1521420cd1" member_type: VOTER last_known_addr { host: "127.11.254.189" port: 36155 } } peers { permanent_uuid: "7e4700f28485420fbba29f2e545c71df" member_type: VOTER last_known_addr { host: "127.11.254.188" port: 36857 } }
I20260812 21:03:59.491994 24462 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7e4700f28485420fbba29f2e545c71df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:03:59.492017 24462 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7e4700f28485420fbba29f2e545c71df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e4700f28485420fbba29f2e545c71df, State: Initialized, Role: FOLLOWER
I20260812 21:03:59.492129 24462 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7e4700f28485420fbba29f2e545c71df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c90e0f61a5354abbbf96692370a65ebd" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 43007 } } peers { permanent_uuid: "5026b3627f2e49f8b4e7cf1521420cd1" member_type: VOTER last_known_addr { host: "127.11.254.189" port: 36155 } } peers { permanent_uuid: "7e4700f28485420fbba29f2e545c71df" member_type: VOTER last_known_addr { host: "127.11.254.188" port: 36857 } }
I20260812 21:03:59.492513 24462 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7e4700f28485420fbba29f2e545c71df [sys.catalog]: configured and running, proceeding with master startup.
I20260812 21:03:59.492503 24475 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e4700f28485420fbba29f2e545c71df [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c90e0f61a5354abbbf96692370a65ebd" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 43007 } } peers { permanent_uuid: "5026b3627f2e49f8b4e7cf1521420cd1" member_type: VOTER last_known_addr { host: "127.11.254.189" port: 36155 } } peers { permanent_uuid: "7e4700f28485420fbba29f2e545c71df" member_type: VOTER last_known_addr { host: "127.11.254.188" port: 36857 } } }
I20260812 21:03:59.492674 24475 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e4700f28485420fbba29f2e545c71df [sys.catalog]: This master's current role is: FOLLOWER
W20260812 21:03:59.494906 24492 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7e4700f28485420fbba29f2e545c71df: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 21:03:59.494992 24492 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 21:03:59.497354 24353 master_runner.cc:428] Detected that this master 7e4700f28485420fbba29f2e545c71df is joining an existing cluster
I20260812 21:03:59.497428 24353 master_runner.cc:432] Initiating AddMaster RPC to add 127.11.254.188:36857
I20260812 21:03:59.506021 24469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:59.499310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32964 (local address 127.11.254.188:36857)
0812 21:03:59.499378 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0812 21:03:59.499380 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:03:59.499547 (+ 167us) server_negotiation.cc:407] Connection header received
0812 21:03:59.499621 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:03:59.499622 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:03:59.499633 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:03:59.499686 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:03:59.500225 (+ 539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:59.500828 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:59.501884 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:59.502207 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:59.502276 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:59.503464 (+ 1188us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:03:59.503465 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:03:59.503465 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:03:59.503472 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:03:59.504763 (+ 1291us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:59.505182 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:59.505183 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:59.505184 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:59.505238 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:59.505449 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:59.505449 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:59.505450 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:59.505551 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:03:59.505601 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0812 21:03:59.505802 (+ 201us) server_negotiation.cc:299] Negotiation successful
0812 21:03:59.505888 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260812 21:03:59.506353 24503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:59.499126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.188:36857 (local address 127.0.0.1:32964)
0812 21:03:59.499481 (+ 355us) negotiation.cc:107] Waiting for socket to connect
0812 21:03:59.499493 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 21:03:59.499566 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:03:59.499739 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:03:59.499740 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:03:59.499920 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:03:59.500147 (+ 227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:59.500149 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:59.500913 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:59.500914 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:59.501790 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:59.501793 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:59.502289 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:59.502290 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:59.502359 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:59.502862 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:03:59.502867 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:03:59.503330 (+ 463us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:03:59.504847 (+ 1517us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:03:59.504848 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:03:59.504848 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:03:59.505085 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:03:59.505306 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:03:59.505306 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:03:59.505307 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:03:59.505388 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:03:59.505608 (+ 220us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:03:59.505609 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:03:59.505663 (+ 54us) client_negotiation.cc:769] Sending connection context
0812 21:03:59.506155 (+ 492us) client_negotiation.cc:240] Negotiation successful
0812 21:03:59.506255 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":302,"spinlock_wait_cycles":9088,"thread_start_us":103,"threads_started":1}
I20260812 21:03:59.513007 24345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:59.499462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41302 (local address 127.11.254.189:36155)
0812 21:03:59.499514 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0812 21:03:59.499516 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:03:59.506815 (+ 7299us) server_negotiation.cc:407] Connection header received
0812 21:03:59.506870 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:03:59.506871 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:03:59.506884 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:03:59.506947 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:03:59.507437 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:59.507932 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:59.508923 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:59.509218 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:59.509274 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:59.510621 (+ 1347us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:03:59.510622 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:03:59.510623 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:03:59.510630 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:03:59.511870 (+ 1240us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:59.512279 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:59.512280 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:59.512280 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:59.512326 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:59.512590 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:59.512591 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:59.512591 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:59.512684 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:03:59.512729 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0812 21:03:59.512877 (+ 148us) server_negotiation.cc:299] Negotiation successful
0812 21:03:59.512938 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260812 21:03:59.513078 24503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:59.499643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.189:36155 (local address 127.0.0.1:41302)
0812 21:03:59.506745 (+ 7102us) negotiation.cc:107] Waiting for socket to connect
0812 21:03:59.506756 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 21:03:59.506809 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:03:59.506950 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:03:59.506951 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:03:59.507194 (+ 243us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:03:59.507339 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:59.507342 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:59.508011 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:59.508012 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:59.508801 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:59.508804 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:59.509369 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:59.509369 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:59.509508 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:59.509945 (+ 437us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:03:59.509979 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:03:59.510459 (+ 480us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:03:59.511953 (+ 1494us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:03:59.511954 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:03:59.511955 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:03:59.512195 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:03:59.512389 (+ 194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:03:59.512390 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:03:59.512390 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:03:59.512530 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:03:59.512739 (+ 209us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:03:59.512740 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:03:59.512789 (+ 49us) client_negotiation.cc:769] Sending connection context
0812 21:03:59.512864 (+ 75us) client_negotiation.cc:240] Negotiation successful
0812 21:03:59.512972 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7090,"thread_start_us":77,"threads_started":1}
I20260812 21:03:59.528261 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 23324
I20260812 21:03:59.542722 23283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:59.499226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38750 (local address 127.11.254.190:43007)
0812 21:03:59.499276 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0812 21:03:59.499278 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:03:59.499877 (+ 599us) server_negotiation.cc:407] Connection header received
0812 21:03:59.499996 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:03:59.499997 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:03:59.500009 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:03:59.500042 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:03:59.500754 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:59.501292 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:59.502369 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:59.502574 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:59.502626 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:59.539647 (+ 37021us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:03:59.539650 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:03:59.539650 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:03:59.539662 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:03:59.541371 (+ 1709us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:59.541816 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:59.541817 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:59.541817 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:59.541882 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:59.542118 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:59.542119 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:59.542119 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:59.542281 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:03:59.542352 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0812 21:03:59.542548 (+ 196us) server_negotiation.cc:299] Negotiation successful
0812 21:03:59.542629 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260812 21:03:59.542738 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.254.129:35727
--local_ip_for_outbound_sockets=127.11.254.129
--tserver_master_addrs=127.11.254.190:43007
--webserver_port=37093
--webserver_interface=127.11.254.129
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/krb5kdc/oryx_127.11.254.129.keytab
--principal=oryx/127.11.254.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.254.148:45807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.11.254.190:43007,127.11.254.189:36155
--tserver_master_addrs=127.11.254.190:43007,127.11.254.189:36155,127.11.254.188:36857 with env {KRB5CCNAME=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 21:03:59.543007 24504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:59.499128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:43007 (local address 127.0.0.1:38750)
0812 21:03:59.499809 (+ 681us) negotiation.cc:107] Waiting for socket to connect
0812 21:03:59.499819 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 21:03:59.499907 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:03:59.500112 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:03:59.500113 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:03:59.500533 (+ 420us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:03:59.500656 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:59.500660 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:59.501358 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:59.501359 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:59.502302 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:59.502305 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:59.502678 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:59.502679 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:59.502804 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:59.538924 (+ 36120us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:03:59.538932 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:03:59.539492 (+ 560us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:03:59.541485 (+ 1993us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:03:59.541486 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:03:59.541487 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:03:59.541733 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:03:59.541975 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:03:59.541976 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:03:59.541977 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:03:59.542051 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:03:59.542354 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:03:59.542355 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:03:59.542415 (+ 60us) client_negotiation.cc:769] Sending connection context
0812 21:03:59.542476 (+ 61us) client_negotiation.cc:240] Negotiation successful
0812 21:03:59.542620 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":640,"mutex_wait_us":2,"thread_start_us":113,"threads_started":1}
Aug 12 21:03:59 dist-test-slave-vpvm krb5kdc[23147](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568638, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.254.190@KRBTEST.COM for oryx/127.11.254.188@KRBTEST.COM
I20260812 21:03:59.554615 24469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:59.545525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32968 (local address 127.11.254.188:36857)
0812 21:03:59.545605 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0812 21:03:59.545608 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 21:03:59.545616 (+ 8us) server_negotiation.cc:407] Connection header received
0812 21:03:59.545651 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:03:59.545652 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:03:59.545664 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:03:59.545726 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:03:59.546082 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:59.546794 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:59.547770 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:59.548075 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:59.548148 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:59.551733 (+ 3585us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:03:59.551735 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:03:59.551735 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:03:59.551743 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:03:59.553310 (+ 1567us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:59.553735 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:59.553736 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:59.553737 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:59.553801 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:59.554022 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:59.554022 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:59.554023 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:59.554186 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:03:59.554260 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0812 21:03:59.554434 (+ 174us) server_negotiation.cc:299] Negotiation successful
0812 21:03:59.554519 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260812 21:03:59.554621 24173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:59.545475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.188:36857 (local address 127.0.0.1:32968)
0812 21:03:59.545546 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0812 21:03:59.545557 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 21:03:59.545594 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:03:59.545747 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:03:59.545748 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:03:59.545922 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:03:59.546003 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:59.546006 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:59.546880 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:59.546882 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:59.547664 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:59.547667 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:59.548155 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:59.548156 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:59.548224 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:59.549830 (+ 1606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:03:59.549836 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:03:59.551612 (+ 1776us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:03:59.553406 (+ 1794us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:03:59.553408 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:03:59.553408 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:03:59.553656 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:03:59.553868 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:03:59.553869 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:03:59.553869 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:03:59.553938 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:03:59.554255 (+ 317us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:03:59.554256 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:03:59.554313 (+ 57us) client_negotiation.cc:769] Sending connection context
0812 21:03:59.554378 (+ 65us) client_negotiation.cc:240] Negotiation successful
0812 21:03:59.554511 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260812 21:03:59.555869 23224 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.11.254.188:36857: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "7e4700f28485420fbba29f2e545c71df"
member_type: NON_VOTER
last_known_addr {
host: "127.11.254.188"
port: 36857
}
attrs {
promote: true
}
}
dest_uuid: "c90e0f61a5354abbbf96692370a65ebd"
cas_config_opid_index: 5
I20260812 21:03:59.556143 23224 consensus_queue.cc:237] T 00000000000000000000000000000000 P c90e0f61a5354abbbf96692370a65ebd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c90e0f61a5354abbbf96692370a65ebd" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 43007 } } peers { permanent_uuid: "5026b3627f2e49f8b4e7cf1521420cd1" member_type: NON_VOTER last_known_addr { host: "127.11.254.189" port: 36155 } attrs { promote: true } } peers { permanent_uuid: "7e4700f28485420fbba29f2e545c71df" member_type: NON_VOTER last_known_addr { host: "127.11.254.188" port: 36857 } attrs { promote: true } }
I20260812 21:03:59.556529 24175 raft_consensus.cc:2932] T 00000000000000000000000000000000 P c90e0f61a5354abbbf96692370a65ebd [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 7e4700f28485420fbba29f2e545c71df (127.11.254.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c90e0f61a5354abbbf96692370a65ebd" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 43007 } } peers { permanent_uuid: "5026b3627f2e49f8b4e7cf1521420cd1" member_type: NON_VOTER last_known_addr { host: "127.11.254.189" port: 36155 } attrs { promote: true } } peers { permanent_uuid: "7e4700f28485420fbba29f2e545c71df" member_type: NON_VOTER last_known_addr { host: "127.11.254.188" port: 36857 } attrs { promote: true } } }
I20260812 21:03:59.557675 24175 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.11.254.188:36857
I20260812 21:03:59.557756 24175 sys_catalog.cc:455] T 00000000000000000000000000000000 P c90e0f61a5354abbbf96692370a65ebd [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "c90e0f61a5354abbbf96692370a65ebd" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c90e0f61a5354abbbf96692370a65ebd" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 43007 } } peers { permanent_uuid: "5026b3627f2e49f8b4e7cf1521420cd1" member_type: NON_VOTER last_known_addr { host: "127.11.254.189" port: 36155 } attrs { promote: true } } peers { permanent_uuid: "7e4700f28485420fbba29f2e545c71df" member_type: NON_VOTER last_known_addr { host: "127.11.254.188" port: 36857 } attrs { promote: true } } }
I20260812 21:03:59.557826 24175 sys_catalog.cc:458] T 00000000000000000000000000000000 P c90e0f61a5354abbbf96692370a65ebd [sys.catalog]: This master's current role is: LEADER
I20260812 21:03:59.558436 24412 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7e4700f28485420fbba29f2e545c71df [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:03:59.559608 24412 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7e4700f28485420fbba29f2e545c71df [term 1 FOLLOWER]: Refusing update from remote peer c90e0f61a5354abbbf96692370a65ebd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260812 21:03:59.559836 24353 master.cc:537] Master@127.11.254.188:36857 shutting down...
I20260812 21:03:59.560923 24174 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c90e0f61a5354abbbf96692370a65ebd [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e4700f28485420fbba29f2e545c71df" member_type: NON_VOTER last_known_addr { host: "127.11.254.188" port: 36857 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 21:03:59.562602 24345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:59.557969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41318 (local address 127.11.254.189:36155)
0812 21:03:59.558166 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0812 21:03:59.558168 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:03:59.558282 (+ 114us) server_negotiation.cc:407] Connection header received
0812 21:03:59.558306 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:03:59.558308 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:03:59.558330 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:03:59.558380 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:03:59.558546 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:59.559475 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:59.561260 (+ 1785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:59.561851 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:59.561920 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:59.562134 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0812 21:03:59.562497 (+ 363us) server_negotiation.cc:299] Negotiation successful
0812 21:03:59.562537 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166}
I20260812 21:03:59.562829 24173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:59.556641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.189:36155 (local address 127.0.0.1:41318)
0812 21:03:59.558195 (+ 1554us) negotiation.cc:107] Waiting for socket to connect
0812 21:03:59.558204 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 21:03:59.558239 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:03:59.558435 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:03:59.558436 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:03:59.558438 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:03:59.558483 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:59.558485 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:59.559565 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:59.559566 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:59.561161 (+ 1595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:59.561165 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:59.562169 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:59.562170 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:59.562243 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:59.562270 (+ 27us) client_negotiation.cc:769] Sending connection context
0812 21:03:59.562319 (+ 49us) client_negotiation.cc:240] Negotiation successful
0812 21:03:59.562360 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1540}
I20260812 21:03:59.563583 24262 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5026b3627f2e49f8b4e7cf1521420cd1 [term 1 LEARNER]: Refusing update from remote peer c90e0f61a5354abbbf96692370a65ebd: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
W20260812 21:03:59.563589 23198 consensus_peers.cc:487] T 00000000000000000000000000000000 P c90e0f61a5354abbbf96692370a65ebd -> Peer 7e4700f28485420fbba29f2e545c71df (127.11.254.188:36857): Couldn't send request to peer 7e4700f28485420fbba29f2e545c71df. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260812 21:03:59.563812 24174 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c90e0f61a5354abbbf96692370a65ebd [LEADER]: Connected to new peer: Peer: permanent_uuid: "5026b3627f2e49f8b4e7cf1521420cd1" member_type: NON_VOTER last_known_addr { host: "127.11.254.189" port: 36155 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260812 21:03:59.564529 24262 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 5026b3627f2e49f8b4e7cf1521420cd1 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 7e4700f28485420fbba29f2e545c71df (127.11.254.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c90e0f61a5354abbbf96692370a65ebd" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 43007 } } peers { permanent_uuid: "5026b3627f2e49f8b4e7cf1521420cd1" member_type: NON_VOTER last_known_addr { host: "127.11.254.189" port: 36155 } attrs { promote: true } } peers { permanent_uuid: "7e4700f28485420fbba29f2e545c71df" member_type: NON_VOTER last_known_addr { host: "127.11.254.188" port: 36857 } attrs { promote: true } } }
I20260812 21:03:59.565860 24174 raft_consensus.cc:1050] T 00000000000000000000000000000000 P c90e0f61a5354abbbf96692370a65ebd: attempting to promote NON_VOTER 5026b3627f2e49f8b4e7cf1521420cd1 to VOTER
I20260812 21:03:59.565920 24316 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5026b3627f2e49f8b4e7cf1521420cd1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c90e0f61a5354abbbf96692370a65ebd. Latest consensus state: current_term: 1 leader_uuid: "c90e0f61a5354abbbf96692370a65ebd" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c90e0f61a5354abbbf96692370a65ebd" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 43007 } } peers { permanent_uuid: "5026b3627f2e49f8b4e7cf1521420cd1" member_type: NON_VOTER last_known_addr { host: "127.11.254.189" port: 36155 } attrs { promote: true } } peers { permanent_uuid: "7e4700f28485420fbba29f2e545c71df" member_type: NON_VOTER last_known_addr { host: "127.11.254.188" port: 36857 } attrs { promote: true } } }
I20260812 21:03:59.566037 24316 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5026b3627f2e49f8b4e7cf1521420cd1 [sys.catalog]: This master's current role is: LEARNER
I20260812 21:03:59.566073 24316 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5026b3627f2e49f8b4e7cf1521420cd1 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c90e0f61a5354abbbf96692370a65ebd" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c90e0f61a5354abbbf96692370a65ebd" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 43007 } } peers { permanent_uuid: "5026b3627f2e49f8b4e7cf1521420cd1" member_type: NON_VOTER last_known_addr { host: "127.11.254.189" port: 36155 } attrs { promote: true } } peers { permanent_uuid: "7e4700f28485420fbba29f2e545c71df" member_type: NON_VOTER last_known_addr { host: "127.11.254.188" port: 36857 } attrs { promote: true } } }
I20260812 21:03:59.566134 24316 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5026b3627f2e49f8b4e7cf1521420cd1 [sys.catalog]: This master's current role is: LEARNER
I20260812 21:03:59.566190 24174 consensus_queue.cc:237] T 00000000000000000000000000000000 P c90e0f61a5354abbbf96692370a65ebd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c90e0f61a5354abbbf96692370a65ebd" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 43007 } } peers { permanent_uuid: "5026b3627f2e49f8b4e7cf1521420cd1" member_type: VOTER last_known_addr { host: "127.11.254.189" port: 36155 } attrs { promote: false } } peers { permanent_uuid: "7e4700f28485420fbba29f2e545c71df" member_type: NON_VOTER last_known_addr { host: "127.11.254.188" port: 36857 } attrs { promote: true } }
W20260812 21:03:59.566763 23198 consensus_peers.cc:487] T 00000000000000000000000000000000 P c90e0f61a5354abbbf96692370a65ebd -> Peer 7e4700f28485420fbba29f2e545c71df (127.11.254.188:36857): Couldn't send request to peer 7e4700f28485420fbba29f2e545c71df. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260812 21:03:59.566812 24262 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5026b3627f2e49f8b4e7cf1521420cd1 [term 1 LEARNER]: Refusing update from remote peer c90e0f61a5354abbbf96692370a65ebd: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260812 21:03:59.567222 24353 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 7e4700f28485420fbba29f2e545c71df [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 21:03:59.567323 24353 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 7e4700f28485420fbba29f2e545c71df [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 21:03:59.567375 24353 tablet_replica.cc:330] T 00000000000000000000000000000000 P 7e4700f28485420fbba29f2e545c71df: stopping tablet replica
I20260812 21:03:59.567562 24513 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c90e0f61a5354abbbf96692370a65ebd [LEADER]: Connected to new peer: Peer: permanent_uuid: "5026b3627f2e49f8b4e7cf1521420cd1" member_type: VOTER last_known_addr { host: "127.11.254.189" port: 36155 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260812 21:03:59.568840 24174 raft_consensus.cc:2932] T 00000000000000000000000000000000 P c90e0f61a5354abbbf96692370a65ebd [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 5026b3627f2e49f8b4e7cf1521420cd1 (127.11.254.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c90e0f61a5354abbbf96692370a65ebd" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 43007 } } peers { permanent_uuid: "5026b3627f2e49f8b4e7cf1521420cd1" member_type: VOTER last_known_addr { host: "127.11.254.189" port: 36155 } attrs { promote: false } } peers { permanent_uuid: "7e4700f28485420fbba29f2e545c71df" member_type: NON_VOTER last_known_addr { host: "127.11.254.188" port: 36857 } attrs { promote: true } } }
I20260812 21:03:59.569855 24174 sys_catalog.cc:455] T 00000000000000000000000000000000 P c90e0f61a5354abbbf96692370a65ebd [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "c90e0f61a5354abbbf96692370a65ebd" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c90e0f61a5354abbbf96692370a65ebd" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 43007 } } peers { permanent_uuid: "5026b3627f2e49f8b4e7cf1521420cd1" member_type: VOTER last_known_addr { host: "127.11.254.189" port: 36155 } attrs { promote: false } } peers { permanent_uuid: "7e4700f28485420fbba29f2e545c71df" member_type: NON_VOTER last_known_addr { host: "127.11.254.188" port: 36857 } attrs { promote: true } } }
I20260812 21:03:59.569937 24174 sys_catalog.cc:458] T 00000000000000000000000000000000 P c90e0f61a5354abbbf96692370a65ebd [sys.catalog]: This master's current role is: LEADER
I20260812 21:03:59.570102 24175 sys_catalog.cc:455] T 00000000000000000000000000000000 P c90e0f61a5354abbbf96692370a65ebd [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c90e0f61a5354abbbf96692370a65ebd" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c90e0f61a5354abbbf96692370a65ebd" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 43007 } } peers { permanent_uuid: "5026b3627f2e49f8b4e7cf1521420cd1" member_type: VOTER last_known_addr { host: "127.11.254.189" port: 36155 } attrs { promote: false } } peers { permanent_uuid: "7e4700f28485420fbba29f2e545c71df" member_type: NON_VOTER last_known_addr { host: "127.11.254.188" port: 36857 } attrs { promote: true } } }
I20260812 21:03:59.570185 24175 sys_catalog.cc:458] T 00000000000000000000000000000000 P c90e0f61a5354abbbf96692370a65ebd [sys.catalog]: This master's current role is: LEADER
I20260812 21:03:59.570237 24262 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 5026b3627f2e49f8b4e7cf1521420cd1 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 5026b3627f2e49f8b4e7cf1521420cd1 (127.11.254.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c90e0f61a5354abbbf96692370a65ebd" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 43007 } } peers { permanent_uuid: "5026b3627f2e49f8b4e7cf1521420cd1" member_type: VOTER last_known_addr { host: "127.11.254.189" port: 36155 } attrs { promote: false } } peers { permanent_uuid: "7e4700f28485420fbba29f2e545c71df" member_type: NON_VOTER last_known_addr { host: "127.11.254.188" port: 36857 } attrs { promote: true } } }
I20260812 21:03:59.571225 24512 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5026b3627f2e49f8b4e7cf1521420cd1 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c90e0f61a5354abbbf96692370a65ebd" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c90e0f61a5354abbbf96692370a65ebd" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 43007 } } peers { permanent_uuid: "5026b3627f2e49f8b4e7cf1521420cd1" member_type: VOTER last_known_addr { host: "127.11.254.189" port: 36155 } attrs { promote: false } } peers { permanent_uuid: "7e4700f28485420fbba29f2e545c71df" member_type: NON_VOTER last_known_addr { host: "127.11.254.188" port: 36857 } attrs { promote: true } } }
I20260812 21:03:59.571323 24512 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5026b3627f2e49f8b4e7cf1521420cd1 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 21:03:59.580639 24353 master.cc:552] Master@127.11.254.188:36857 shutdown complete.
I20260812 21:03:59.580705 24353 master_runner.cc:305] Clearing existing system tablet
I20260812 21:03:59.580829 24353 fs_manager.cc:359] Metadata directory not provided
I20260812 21:03:59.580876 24353 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:03:59.581629 24353 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260812 21:03:59.582175 24521 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:03:59.582322 24353 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:03:59.582386 24353 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 21:03:59.582406 24353 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/master-2/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/master-2/wal
uuid: "7e4700f28485420fbba29f2e545c71df"
format_stamp: "Formatted at 2026-08-12 21:03:59 on dist-test-slave-vpvm"
server_key: "9945d4ff1f12643293ae9659102aa78e"
server_key_iv: "775a9a0e271a356a1150bab9644e76f5"
server_key_version: "encryptionkey@0"
I20260812 21:03:59.583115 24353 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 7e4700f28485420fbba29f2e545c71df: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 21:03:59.584641 24353 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 7e4700f28485420fbba29f2e545c71df: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260812 21:03:59.584712 24353 log.cc:1196] T 00000000000000000000000000000000 P 7e4700f28485420fbba29f2e545c71df: Deleting WAL directory at /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260812 21:03:59.584903 24353 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 7e4700f28485420fbba29f2e545c71df: Deleting consensus metadata
I20260812 21:03:59.585016 24353 master_runner.cc:315] Copying system tablet from 127.11.254.190:43007
I20260812 21:03:59.585640 24353 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 7e4700f28485420fbba29f2e545c71df: tablet copy: Beginning tablet copy session from remote peer at address 127.11.254.190:43007
I20260812 21:03:59.593139 24530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:59.586023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:43007 (local address 127.0.0.1:38764)
0812 21:03:59.586150 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0812 21:03:59.586161 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 21:03:59.586230 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:03:59.586385 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:03:59.586386 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:03:59.586595 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:03:59.586665 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:59.586667 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:59.587636 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:59.587638 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:59.588435 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:59.588438 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:59.588923 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:59.588924 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:59.589001 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:59.589505 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:03:59.589510 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:03:59.589923 (+ 413us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:03:59.591880 (+ 1957us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:03:59.591881 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:03:59.591881 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:03:59.592110 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:03:59.592414 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:03:59.592414 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:03:59.592415 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:03:59.592497 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:03:59.592768 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:03:59.592769 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:03:59.592833 (+ 64us) client_negotiation.cc:769] Sending connection context
0812 21:03:59.592913 (+ 80us) client_negotiation.cc:240] Negotiation successful
0812 21:03:59.593033 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":69,"threads_started":1}
I20260812 21:03:59.593194 23283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:59.586082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38764 (local address 127.11.254.190:43007)
0812 21:03:59.586131 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0812 21:03:59.586133 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:03:59.586231 (+ 98us) server_negotiation.cc:407] Connection header received
0812 21:03:59.586285 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:03:59.586286 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:03:59.586297 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:03:59.586371 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:03:59.586744 (+ 373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:59.587542 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:59.588534 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:59.588833 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:59.588904 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:59.590060 (+ 1156us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:03:59.590062 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:03:59.590062 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:03:59.590071 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:03:59.591785 (+ 1714us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:59.592231 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:59.592233 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:59.592233 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:59.592298 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:59.592565 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:59.592566 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:59.592567 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:59.592702 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:03:59.592766 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0812 21:03:59.592982 (+ 216us) server_negotiation.cc:299] Negotiation successful
0812 21:03:59.593104 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260812 21:03:59.594048 23257 tablet_copy_service.cc:140] P c90e0f61a5354abbbf96692370a65ebd: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 7e4700f28485420fbba29f2e545c71df ({username='oryx', principal='oryx/127.11.254.188@KRBTEST.COM'} at 127.0.0.1:38764)
I20260812 21:03:59.594126 23257 tablet_copy_service.cc:161] P c90e0f61a5354abbbf96692370a65ebd: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 7e4700f28485420fbba29f2e545c71df at {username='oryx', principal='oryx/127.11.254.188@KRBTEST.COM'} at 127.0.0.1:38764: session id = 7e4700f28485420fbba29f2e545c71df-00000000000000000000000000000000
I20260812 21:03:59.594792 23257 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P c90e0f61a5354abbbf96692370a65ebd: Tablet Copy: opened 0 blocks and 1 log segments
I20260812 21:03:59.595338 24353 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260812 21:03:59.596774 24353 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 7e4700f28485420fbba29f2e545c71df: tablet copy: Starting download of 0 data blocks...
I20260812 21:03:59.596988 24353 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 7e4700f28485420fbba29f2e545c71df: tablet copy: Starting download of 1 WAL segments...
I20260812 21:03:59.597882 24353 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 7e4700f28485420fbba29f2e545c71df: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260812 21:03:59.598975 23257 tablet_copy_service.cc:342] P c90e0f61a5354abbbf96692370a65ebd: Request end of tablet copy session 7e4700f28485420fbba29f2e545c71df-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.11.254.188@KRBTEST.COM'} at 127.0.0.1:38764
I20260812 21:03:59.599045 23257 tablet_copy_service.cc:433] P c90e0f61a5354abbbf96692370a65ebd: ending tablet copy session 7e4700f28485420fbba29f2e545c71df-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 7e4700f28485420fbba29f2e545c71df
I20260812 21:03:59.602494 24353 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:03:59.603968 24533 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:03:59.604172 24534 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:03:59.604210 24536 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:03:59.604429 24353 server_base.cc:1034] running on GCE node
I20260812 21:03:59.604667 24353 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:03:59.604827 24353 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:03:59.605983 24353 hybrid_clock.cc:648] HybridClock initialized: now 1786568639605998 us; error 55 us; skew 500 ppm
Aug 12 21:03:59 dist-test-slave-vpvm krb5kdc[23147](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568639, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.254.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:03:59.607559 24353 init.cc:377] Logged in from keytab as oryx/127.11.254.188@KRBTEST.COM (short username oryx)
I20260812 21:03:59.607854 24353 webserver.cc:460] Webserver started at http://127.11.254.188:35385/ using document root <none> and password file <none>
I20260812 21:03:59.607986 24353 fs_manager.cc:359] Metadata directory not provided
I20260812 21:03:59.608024 24353 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:03:59.608817 24353 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 21:03:59.609441 24552 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:03:59.609628 24353 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 21:03:59.609686 24353 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/master-2/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/master-2/wal
uuid: "7e4700f28485420fbba29f2e545c71df"
format_stamp: "Formatted at 2026-08-12 21:03:59 on dist-test-slave-vpvm"
server_key: "9945d4ff1f12643293ae9659102aa78e"
server_key_iv: "775a9a0e271a356a1150bab9644e76f5"
server_key_version: "encryptionkey@0"
I20260812 21:03:59.609751 24353 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:03:59.622112 24353 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:03:59.624001 24353 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:03:59.627228 24353 rpc_server.cc:307] RPC server started. Bound to: 127.11.254.188:36857
I20260812 21:03:59.627547 24638 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.254.188:36857 every 8 connection(s)
I20260812 21:03:59.627589 24353 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/master-2/data/info.pb
I20260812 21:03:59.628713 24640 sys_catalog.cc:263] Verifying existing consensus state
I20260812 21:03:59.629269 24640 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e4700f28485420fbba29f2e545c71df: Bootstrap starting.
I20260812 21:03:59.632697 24640 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e4700f28485420fbba29f2e545c71df: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 21:03:59.632979 24640 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e4700f28485420fbba29f2e545c71df: Bootstrap complete.
I20260812 21:03:59.633185 24640 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7e4700f28485420fbba29f2e545c71df [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c90e0f61a5354abbbf96692370a65ebd" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 43007 } } peers { permanent_uuid: "5026b3627f2e49f8b4e7cf1521420cd1" member_type: VOTER last_known_addr { host: "127.11.254.189" port: 36155 } attrs { promote: false } } peers { permanent_uuid: "7e4700f28485420fbba29f2e545c71df" member_type: NON_VOTER last_known_addr { host: "127.11.254.188" port: 36857 } attrs { promote: true } }
I20260812 21:03:59.633279 24640 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7e4700f28485420fbba29f2e545c71df [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 7e4700f28485420fbba29f2e545c71df, State: Initialized, Role: LEARNER
I20260812 21:03:59.633315 24640 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7e4700f28485420fbba29f2e545c71df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c90e0f61a5354abbbf96692370a65ebd" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 43007 } } peers { permanent_uuid: "5026b3627f2e49f8b4e7cf1521420cd1" member_type: VOTER last_known_addr { host: "127.11.254.189" port: 36155 } attrs { promote: false } } peers { permanent_uuid: "7e4700f28485420fbba29f2e545c71df" member_type: NON_VOTER last_known_addr { host: "127.11.254.188" port: 36857 } attrs { promote: true } }
I20260812 21:03:59.633528 24640 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7e4700f28485420fbba29f2e545c71df [sys.catalog]: configured and running, proceeding with master startup.
I20260812 21:03:59.635435 24661 catalog_manager.cc:1259] Loaded cluster ID: b39e89298b2f46b5a2e07cc9a4507645
I20260812 21:03:59.635550 24661 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 7e4700f28485420fbba29f2e545c71df: loading cluster ID for follower catalog manager: success
I20260812 21:03:59.636731 24661 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 7e4700f28485420fbba29f2e545c71df: acquiring CA information for follower catalog manager: success
I20260812 21:03:59.637166 24661 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 7e4700f28485420fbba29f2e545c71df: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 21:03:59.638065 24643 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e4700f28485420fbba29f2e545c71df [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c90e0f61a5354abbbf96692370a65ebd" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 43007 } } peers { permanent_uuid: "5026b3627f2e49f8b4e7cf1521420cd1" member_type: VOTER last_known_addr { host: "127.11.254.189" port: 36155 } attrs { promote: false } } peers { permanent_uuid: "7e4700f28485420fbba29f2e545c71df" member_type: NON_VOTER last_known_addr { host: "127.11.254.188" port: 36857 } attrs { promote: true } } }
I20260812 21:03:59.638216 24643 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e4700f28485420fbba29f2e545c71df [sys.catalog]: This master's current role is: LEARNER
W20260812 21:03:59.651911 24509 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:03:59.652177 24509 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:03:59.652218 24509 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:03:59.653558 24509 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:03:59.653609 24509 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:03:59.653651 24509 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.254.129
I20260812 21:03:59.655570 24509 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.254.148:45807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/krb5kdc/oryx_127.11.254.129.keytab
--principal=oryx/127.11.254.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.254.129:35727
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.254.129
--webserver_port=37093
--tserver_master_addrs=127.11.254.190:43007,127.11.254.189:36155,127.11.254.188:36857
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24509
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.254.129
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:03:59.655866 24509 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:03:59.656124 24509 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:03:59.662084 24671 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:03:59.662155 24509 server_base.cc:1034] running on GCE node
W20260812 21:03:59.662091 24669 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:03:59.662161 24674 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:03:59.662510 24509 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:03:59.662883 24509 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:03:59.664057 24509 hybrid_clock.cc:648] HybridClock initialized: now 1786568639664036 us; error 35 us; skew 500 ppm
Aug 12 21:03:59 dist-test-slave-vpvm krb5kdc[23147](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568639, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.254.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:03:59.666759 24509 init.cc:377] Logged in from keytab as oryx/127.11.254.129@KRBTEST.COM (short username oryx)
I20260812 21:03:59.667299 24509 webserver.cc:460] Webserver started at http://127.11.254.129:37093/ using document root <none> and password file <none>
I20260812 21:03:59.667542 24509 fs_manager.cc:359] Metadata directory not provided
I20260812 21:03:59.667606 24509 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:03:59.669019 24509 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 21:03:59.669767 24686 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:03:59.669945 24509 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 21:03:59.670082 24509 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/ts-0/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/ts-0/wal
uuid: "8e16af0efe6c4f698e93d11250de344b"
format_stamp: "Formatted at 2026-08-12 21:03:58 on dist-test-slave-vpvm"
server_key: "5627151f9d2ca36ca770267088374f4c"
server_key_iv: "2de650a821e575b652c94dc69933702d"
server_key_version: "encryptionkey@0"
I20260812 21:03:59.670462 24509 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:03:59.689612 24509 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:03:59.691975 24509 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:03:59.692130 24509 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:03:59.692395 24509 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:03:59.692723 24509 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:03:59.692765 24509 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:03:59.692788 24509 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:03:59.692811 24509 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:03:59.706902 24509 rpc_server.cc:307] RPC server started. Bound to: 127.11.254.129:35727
I20260812 21:03:59.707077 24864 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.254.129:35727 every 8 connection(s)
I20260812 21:03:59.707424 24509 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/ts-0/data/info.pb
I20260812 21:03:59.709699 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 24509
I20260812 21:03:59.709853 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 23543
Aug 12 21:03:59 dist-test-slave-vpvm krb5kdc[23147](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568639, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.254.129@KRBTEST.COM for oryx/127.11.254.189@KRBTEST.COM
I20260812 21:03:59.721551 24345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:59.708293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.129:50403 (local address 127.11.254.189:36155)
0812 21:03:59.708347 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0812 21:03:59.708349 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:03:59.708833 (+ 484us) server_negotiation.cc:407] Connection header received
0812 21:03:59.710412 (+ 1579us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:03:59.710414 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:03:59.710429 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:03:59.710457 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:03:59.711800 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:59.712420 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:59.714052 (+ 1632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:59.714256 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:59.714290 (+ 34us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:59.717509 (+ 3219us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:03:59.717510 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:03:59.717510 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:03:59.717519 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:03:59.719657 (+ 2138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:59.720334 (+ 677us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:59.720335 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:59.720336 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:59.720410 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:59.720752 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:59.720753 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:59.720754 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:59.720941 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:03:59.720993 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0812 21:03:59.721383 (+ 390us) server_negotiation.cc:299] Negotiation successful
0812 21:03:59.721465 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 21:03:59.722401 24878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:59.708278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.189:36155 (local address 127.11.254.129:50403)
0812 21:03:59.708648 (+ 370us) negotiation.cc:107] Waiting for socket to connect
0812 21:03:59.708697 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0812 21:03:59.710285 (+ 1588us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:03:59.710701 (+ 416us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:03:59.710705 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:03:59.711102 (+ 397us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:03:59.711684 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:59.711691 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:59.712611 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:59.712613 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:59.713900 (+ 1287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:59.713903 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:59.714370 (+ 467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:59.714371 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:59.714550 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:59.715459 (+ 909us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:03:59.715477 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:03:59.717403 (+ 1926us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:03:59.719780 (+ 2377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:03:59.719793 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:03:59.719796 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:03:59.720224 (+ 428us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:03:59.720494 (+ 270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:03:59.720495 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:03:59.720495 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:03:59.720662 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:03:59.721051 (+ 389us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:03:59.721060 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:03:59.721251 (+ 191us) client_negotiation.cc:769] Sending connection context
0812 21:03:59.721893 (+ 642us) client_negotiation.cc:240] Negotiation successful
0812 21:03:59.722109 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":301,"mutex_wait_us":75,"thread_start_us":186,"threads_started":1}
I20260812 21:03:59.723094 24869 heartbeater.cc:347] Connected to a master server at 127.11.254.189:36155
I20260812 21:03:59.723205 24869 heartbeater.cc:464] Registering TS with master...
I20260812 21:03:59.723381 24869 heartbeater.cc:511] Master 127.11.254.189:36155 requested a full tablet report, sending...
I20260812 21:03:59.723899 24246 ts_manager.cc:194] Registered new tserver with Master: 8e16af0efe6c4f698e93d11250de344b (127.11.254.129:35727)
I20260812 21:03:59.731843 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.254.130:39443
--local_ip_for_outbound_sockets=127.11.254.130
--tserver_master_addrs=127.11.254.190:43007
--webserver_port=33467
--webserver_interface=127.11.254.130
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/krb5kdc/oryx_127.11.254.130.keytab
--principal=oryx/127.11.254.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.254.148:45807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.11.254.190:43007,127.11.254.189:36155
--tserver_master_addrs=127.11.254.190:43007,127.11.254.189:36155,127.11.254.188:36857 with env {KRB5CCNAME=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 21:03:59 dist-test-slave-vpvm krb5kdc[23147](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568639, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.254.129@KRBTEST.COM for oryx/127.11.254.188@KRBTEST.COM
Aug 12 21:03:59 dist-test-slave-vpvm krb5kdc[23147](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568639, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.254.129@KRBTEST.COM for oryx/127.11.254.190@KRBTEST.COM
I20260812 21:03:59.736131 24876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:59.708278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.188:36857 (local address 127.11.254.129:36723)
0812 21:03:59.708648 (+ 370us) negotiation.cc:107] Waiting for socket to connect
0812 21:03:59.708696 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 21:03:59.726654 (+ 17958us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:03:59.726943 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:03:59.726944 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:03:59.727275 (+ 331us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:03:59.727398 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:59.727401 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:59.728366 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:59.728367 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:59.729401 (+ 1034us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:59.729404 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:59.729753 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:59.729753 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:59.729926 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:59.730839 (+ 913us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:03:59.730844 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:03:59.732995 (+ 2151us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:03:59.734915 (+ 1920us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:03:59.734916 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:03:59.734916 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:03:59.735149 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:03:59.735334 (+ 185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:03:59.735335 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:03:59.735335 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:03:59.735395 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:03:59.735680 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:03:59.735681 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:03:59.735725 (+ 44us) client_negotiation.cc:769] Sending connection context
0812 21:03:59.735786 (+ 61us) client_negotiation.cc:240] Negotiation successful
0812 21:03:59.735902 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":294,"thread_start_us":142,"threads_started":1}
I20260812 21:03:59.736287 23283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:59.708872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.129:50053 (local address 127.11.254.190:43007)
0812 21:03:59.708917 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0812 21:03:59.708919 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:03:59.709187 (+ 268us) server_negotiation.cc:407] Connection header received
0812 21:03:59.727729 (+ 18542us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:03:59.727731 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:03:59.727746 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:03:59.727810 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:03:59.728512 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:59.729277 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:59.730430 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:59.730670 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:59.730723 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:59.733114 (+ 2391us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:03:59.733116 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:03:59.733116 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:03:59.733124 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:03:59.735010 (+ 1886us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:59.735384 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:59.735386 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:59.735386 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:59.735441 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:59.735623 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:59.735623 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:59.735624 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:59.735749 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:03:59.735809 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0812 21:03:59.736108 (+ 299us) server_negotiation.cc:299] Negotiation successful
0812 21:03:59.736183 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 21:03:59.736351 24879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:59.708773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:43007 (local address 127.11.254.129:50053)
0812 21:03:59.709108 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0812 21:03:59.709118 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 21:03:59.727624 (+ 18506us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:03:59.727831 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:03:59.727832 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:03:59.728184 (+ 352us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:03:59.728402 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:59.728405 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:59.729377 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:59.729378 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:59.730334 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:59.730337 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:59.730747 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:59.730748 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:59.730872 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:59.731364 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:03:59.731369 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:03:59.733034 (+ 1665us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:03:59.735080 (+ 2046us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:03:59.735081 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:03:59.735082 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:03:59.735312 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:03:59.735493 (+ 181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:03:59.735494 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:03:59.735494 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:03:59.735559 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:03:59.735810 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:03:59.735811 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:03:59.735858 (+ 47us) client_negotiation.cc:769] Sending connection context
0812 21:03:59.735920 (+ 62us) client_negotiation.cc:240] Negotiation successful
0812 21:03:59.736097 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":113,"threads_started":1}
I20260812 21:03:59.736617 24875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:59.708326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.129:36723 (local address 127.11.254.188:36857)
0812 21:03:59.708500 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0812 21:03:59.708502 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:03:59.708815 (+ 313us) server_negotiation.cc:407] Connection header received
0812 21:03:59.726772 (+ 17957us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:03:59.726774 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:03:59.726790 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:03:59.726921 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:03:59.727496 (+ 575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:59.728289 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:59.729477 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:59.729691 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:59.729748 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:59.733093 (+ 3345us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:03:59.733095 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:03:59.733095 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:03:59.733105 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:03:59.734814 (+ 1709us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:59.735213 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:59.735215 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:59.735215 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:59.735277 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:59.735464 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:59.735465 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:59.735465 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:59.735594 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:03:59.735653 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0812 21:03:59.736340 (+ 687us) server_negotiation.cc:299] Negotiation successful
0812 21:03:59.736532 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":60,"threads_started":1}
I20260812 21:03:59.736763 24867 heartbeater.cc:347] Connected to a master server at 127.11.254.190:43007
I20260812 21:03:59.736820 24867 heartbeater.cc:464] Registering TS with master...
I20260812 21:03:59.736904 24867 heartbeater.cc:511] Master 127.11.254.190:43007 requested a full tablet report, sending...
I20260812 21:03:59.737017 24865 heartbeater.cc:347] Connected to a master server at 127.11.254.188:36857
I20260812 21:03:59.737064 24865 heartbeater.cc:464] Registering TS with master...
I20260812 21:03:59.737133 24865 heartbeater.cc:511] Master 127.11.254.188:36857 requested a full tablet report, sending...
I20260812 21:03:59.737224 23224 ts_manager.cc:194] Re-registered known tserver with Master: 8e16af0efe6c4f698e93d11250de344b (127.11.254.129:35727)
I20260812 21:03:59.737715 23224 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.11.254.129@KRBTEST.COM'} at 127.11.254.129:50053
I20260812 21:03:59.738180 24582 ts_manager.cc:194] Registered new tserver with Master: 8e16af0efe6c4f698e93d11250de344b (127.11.254.129:35727)
W20260812 21:03:59.820209 24880 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:03:59.820487 24880 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:03:59.820520 24880 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:03:59.821717 24880 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:03:59.821763 24880 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:03:59.821798 24880 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.254.130
I20260812 21:03:59.823465 24880 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.254.148:45807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/krb5kdc/oryx_127.11.254.130.keytab
--principal=oryx/127.11.254.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.254.130:39443
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.254.130
--webserver_port=33467
--tserver_master_addrs=127.11.254.190:43007,127.11.254.189:36155,127.11.254.188:36857
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24880
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.254.130
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:03:59.823724 24880 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:03:59.823962 24880 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:03:59.829329 24891 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:03:59.829355 24890 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:03:59.829381 24880 server_base.cc:1034] running on GCE node
W20260812 21:03:59.829351 24894 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:03:59.829695 24880 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:03:59.829938 24880 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:03:59.831125 24880 hybrid_clock.cc:648] HybridClock initialized: now 1786568639831104 us; error 30 us; skew 500 ppm
Aug 12 21:03:59 dist-test-slave-vpvm krb5kdc[23147](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568639, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.254.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:03:59.833783 24880 init.cc:377] Logged in from keytab as oryx/127.11.254.130@KRBTEST.COM (short username oryx)
I20260812 21:03:59.834281 24880 webserver.cc:460] Webserver started at http://127.11.254.130:33467/ using document root <none> and password file <none>
I20260812 21:03:59.834517 24880 fs_manager.cc:359] Metadata directory not provided
I20260812 21:03:59.834595 24880 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:03:59.836020 24880 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.001s
I20260812 21:03:59.836719 24904 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:03:59.836859 24880 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 21:03:59.836925 24880 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/ts-1/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/ts-1/wal
uuid: "a9da4e76358f4ee18755f38b4d8de7ad"
format_stamp: "Formatted at 2026-08-12 21:03:58 on dist-test-slave-vpvm"
server_key: "4e7e5c2df263aff72b84962e259fd96c"
server_key_iv: "a1263a661ad10e300ff92c009df9e0d3"
server_key_version: "encryptionkey@0"
I20260812 21:03:59.837260 24880 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:03:59.850973 24880 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:03:59.853332 24880 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:03:59.853461 24880 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:03:59.853678 24880 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:03:59.854027 24880 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:03:59.854067 24880 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:03:59.854091 24880 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:03:59.854110 24880 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:03:59.861109 24880 rpc_server.cc:307] RPC server started. Bound to: 127.11.254.130:39443
I20260812 21:03:59.861151 25110 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.254.130:39443 every 8 connection(s)
I20260812 21:03:59.861564 24880 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/ts-1/data/info.pb
I20260812 21:03:59.869683 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 24880
I20260812 21:03:59.869867 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 23787
Aug 12 21:03:59 dist-test-slave-vpvm krb5kdc[23147](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568639, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.254.130@KRBTEST.COM for oryx/127.11.254.188@KRBTEST.COM
Aug 12 21:03:59 dist-test-slave-vpvm krb5kdc[23147](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568639, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.254.130@KRBTEST.COM for oryx/127.11.254.190@KRBTEST.COM
Aug 12 21:03:59 dist-test-slave-vpvm krb5kdc[23147](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568639, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.254.130@KRBTEST.COM for oryx/127.11.254.189@KRBTEST.COM
I20260812 21:03:59.879352 24345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:59.862466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.130:43111 (local address 127.11.254.189:36155)
0812 21:03:59.862525 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0812 21:03:59.862527 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:03:59.862955 (+ 428us) server_negotiation.cc:407] Connection header received
0812 21:03:59.864183 (+ 1228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:03:59.864185 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:03:59.864199 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:03:59.864275 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:03:59.865437 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:59.866068 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:59.867477 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:59.867784 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:59.867848 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:59.873146 (+ 5298us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:03:59.873148 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:03:59.873148 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:03:59.873157 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:03:59.877826 (+ 4669us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:59.878365 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:59.878367 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:59.878367 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:59.878422 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:59.878660 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:59.878661 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:59.878661 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:59.878785 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:03:59.878822 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0812 21:03:59.879178 (+ 356us) server_negotiation.cc:299] Negotiation successful
0812 21:03:59.879264 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 21:03:59.879859 25118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:59.862481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.189:36155 (local address 127.11.254.130:43111)
0812 21:03:59.862807 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0812 21:03:59.862851 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 21:03:59.864063 (+ 1212us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:03:59.864392 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:03:59.864396 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:03:59.864944 (+ 548us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:03:59.865326 (+ 382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:59.865350 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:59.866296 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:59.866297 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:59.867382 (+ 1085us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:59.867385 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:59.867865 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:59.867866 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:59.868114 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:59.868843 (+ 729us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:03:59.868848 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:03:59.873008 (+ 4160us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:03:59.877940 (+ 4932us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:03:59.877977 (+ 37us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:03:59.877980 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:03:59.878285 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:03:59.878488 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:03:59.878489 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:03:59.878489 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:03:59.878605 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:03:59.878856 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:03:59.878860 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:03:59.879040 (+ 180us) client_negotiation.cc:769] Sending connection context
0812 21:03:59.879497 (+ 457us) client_negotiation.cc:240] Negotiation successful
0812 21:03:59.879687 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":297,"mutex_wait_us":85,"thread_start_us":237,"threads_started":1}
I20260812 21:03:59.891546 25115 heartbeater.cc:347] Connected to a master server at 127.11.254.189:36155
I20260812 21:03:59.891690 25115 heartbeater.cc:464] Registering TS with master...
I20260812 21:03:59.891922 25115 heartbeater.cc:511] Master 127.11.254.189:36155 requested a full tablet report, sending...
I20260812 21:03:59.892468 24875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:59.862437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.130:38451 (local address 127.11.254.188:36857)
0812 21:03:59.862544 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0812 21:03:59.862547 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 21:03:59.862929 (+ 382us) server_negotiation.cc:407] Connection header received
0812 21:03:59.864207 (+ 1278us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:03:59.864208 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:03:59.864226 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:03:59.864256 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:03:59.865449 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:59.866292 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:59.867623 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:59.867913 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:59.867971 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:59.871577 (+ 3606us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:03:59.871579 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:03:59.871579 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:03:59.871588 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:03:59.891133 (+ 19545us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:59.891642 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:59.891644 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:59.891644 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:59.891716 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:59.891901 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:59.891902 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:59.891902 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:59.892037 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:03:59.892077 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0812 21:03:59.892267 (+ 190us) server_negotiation.cc:299] Negotiation successful
0812 21:03:59.892377 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260812 21:03:59.892470 24246 ts_manager.cc:194] Registered new tserver with Master: a9da4e76358f4ee18755f38b4d8de7ad (127.11.254.130:39443)
I20260812 21:03:59.892845 25117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:59.862445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.188:36857 (local address 127.11.254.130:38451)
0812 21:03:59.862808 (+ 363us) negotiation.cc:107] Waiting for socket to connect
0812 21:03:59.862853 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 21:03:59.864082 (+ 1229us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:03:59.864400 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:03:59.864401 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:03:59.864944 (+ 543us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:03:59.865327 (+ 383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:59.865333 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:59.866385 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:59.866386 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:59.867547 (+ 1161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:59.867550 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:59.868002 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:59.868003 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:59.868113 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:59.868754 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:03:59.868765 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:03:59.871458 (+ 2693us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:03:59.891268 (+ 19810us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:03:59.891269 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:03:59.891270 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:03:59.891528 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:03:59.891785 (+ 257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:03:59.891786 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:03:59.891786 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:03:59.891842 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:03:59.892118 (+ 276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:03:59.892119 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:03:59.892166 (+ 47us) client_negotiation.cc:769] Sending connection context
0812 21:03:59.892584 (+ 418us) client_negotiation.cc:240] Negotiation successful
0812 21:03:59.892718 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":318,"mutex_wait_us":131,"thread_start_us":296,"threads_started":1}
I20260812 21:03:59.893158 23283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:59.863112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.130:60073 (local address 127.11.254.190:43007)
0812 21:03:59.863188 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0812 21:03:59.863190 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:03:59.863248 (+ 58us) server_negotiation.cc:407] Connection header received
0812 21:03:59.864208 (+ 960us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:03:59.864210 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:03:59.864224 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:03:59.864256 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:03:59.865449 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:59.866286 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:59.867722 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:03:59.867970 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:03:59.868038 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:59.873146 (+ 5108us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:03:59.873148 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:03:59.873149 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:03:59.873158 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:03:59.891861 (+ 18703us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:59.892271 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:59.892273 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:59.892273 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:59.892338 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:03:59.892549 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:03:59.892550 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:03:59.892550 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:03:59.892687 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:03:59.892732 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0812 21:03:59.892936 (+ 204us) server_negotiation.cc:299] Negotiation successful
0812 21:03:59.893073 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260812 21:03:59.893188 25113 heartbeater.cc:347] Connected to a master server at 127.11.254.188:36857
I20260812 21:03:59.893236 25113 heartbeater.cc:464] Registering TS with master...
I20260812 21:03:59.893307 25113 heartbeater.cc:511] Master 127.11.254.188:36857 requested a full tablet report, sending...
I20260812 21:03:59.893538 25119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:03:59.862947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:43007 (local address 127.11.254.130:60073)
0812 21:03:59.863192 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0812 21:03:59.863203 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 21:03:59.864095 (+ 892us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:03:59.864440 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:03:59.864441 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:03:59.864944 (+ 503us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:03:59.865327 (+ 383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:59.865333 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:59.866385 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:59.866386 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:59.867600 (+ 1214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:03:59.867602 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:03:59.868101 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:03:59.868102 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:03:59.868158 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:03:59.868851 (+ 693us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:03:59.868855 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:03:59.873030 (+ 4175us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:03:59.891970 (+ 18940us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:03:59.891971 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:03:59.891972 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:03:59.892176 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:03:59.892407 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:03:59.892408 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:03:59.892408 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:03:59.892477 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:03:59.892777 (+ 300us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:03:59.892778 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:03:59.892821 (+ 43us) client_negotiation.cc:769] Sending connection context
0812 21:03:59.893232 (+ 411us) client_negotiation.cc:240] Negotiation successful
0812 21:03:59.893346 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":223,"mutex_wait_us":61,"thread_start_us":65,"threads_started":1}
I20260812 21:03:59.893594 24582 ts_manager.cc:194] Registered new tserver with Master: a9da4e76358f4ee18755f38b4d8de7ad (127.11.254.130:39443)
I20260812 21:03:59.893970 25114 heartbeater.cc:347] Connected to a master server at 127.11.254.190:43007
I20260812 21:03:59.894021 25114 heartbeater.cc:464] Registering TS with master...
I20260812 21:03:59.894080 25114 heartbeater.cc:511] Master 127.11.254.190:43007 requested a full tablet report, sending...
I20260812 21:03:59.894362 23224 ts_manager.cc:194] Re-registered known tserver with Master: a9da4e76358f4ee18755f38b4d8de7ad (127.11.254.130:39443)
I20260812 21:03:59.894846 23224 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.11.254.130@KRBTEST.COM'} at 127.11.254.130:60073
I20260812 21:03:59.895823 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.254.131:46307
--local_ip_for_outbound_sockets=127.11.254.131
--tserver_master_addrs=127.11.254.190:43007
--webserver_port=42619
--webserver_interface=127.11.254.131
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/krb5kdc/oryx_127.11.254.131.keytab
--principal=oryx/127.11.254.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.254.148:45807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.11.254.190:43007,127.11.254.189:36155
--tserver_master_addrs=127.11.254.190:43007,127.11.254.189:36155,127.11.254.188:36857 with env {KRB5CCNAME=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:03:59.981372 25121 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:03:59.981608 25121 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:03:59.981632 25121 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:03:59.982900 25121 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:03:59.982956 25121 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:03:59.982986 25121 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.254.131
I20260812 21:03:59.984665 25121 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.254.148:45807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/krb5kdc/oryx_127.11.254.131.keytab
--principal=oryx/127.11.254.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.254.131:46307
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.254.131
--webserver_port=42619
--tserver_master_addrs=127.11.254.190:43007,127.11.254.189:36155,127.11.254.188:36857
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25121
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.254.131
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:03:59.984937 25121 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:03:59.985209 25121 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:03:59.990633 25131 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:03:59.990658 25134 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:03:59.990720 25121 server_base.cc:1034] running on GCE node
W20260812 21:03:59.990629 25132 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:03:59.991088 25121 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:03:59.991359 25121 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:03:59.992544 25121 hybrid_clock.cc:648] HybridClock initialized: now 1786568639992528 us; error 30 us; skew 500 ppm
Aug 12 21:03:59 dist-test-slave-vpvm krb5kdc[23147](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568639, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.254.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:03:59.995250 25121 init.cc:377] Logged in from keytab as oryx/127.11.254.131@KRBTEST.COM (short username oryx)
I20260812 21:03:59.995739 25121 webserver.cc:460] Webserver started at http://127.11.254.131:42619/ using document root <none> and password file <none>
I20260812 21:03:59.995967 25121 fs_manager.cc:359] Metadata directory not provided
I20260812 21:03:59.996030 25121 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:03:59.997393 25121 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 21:03:59.998106 25146 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:03:59.998288 25121 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 21:03:59.998390 25121 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/ts-2/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/ts-2/wal
uuid: "228262922b09413f8fe37401b0a932bd"
format_stamp: "Formatted at 2026-08-12 21:03:58 on dist-test-slave-vpvm"
server_key: "6ee62bd1ea25c42aecbcefc56e9fd6bc"
server_key_iv: "8674148ee857a2477fe937dd36369f62"
server_key_version: "encryptionkey@0"
I20260812 21:03:59.998811 25121 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:04:00.020821 25121 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:04:00.023214 25121 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:04:00.023372 25121 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:04:00.023650 25121 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:04:00.023958 25121 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:04:00.023993 25121 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:04:00.024024 25121 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:04:00.024041 25121 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:04:00.029942 25121 rpc_server.cc:307] RPC server started. Bound to: 127.11.254.131:46307
I20260812 21:04:00.030028 25337 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.254.131:46307 every 8 connection(s)
I20260812 21:04:00.030368 25121 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/ts-2/data/info.pb
I20260812 21:04:00.032171 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 25121
Aug 12 21:04:00 dist-test-slave-vpvm krb5kdc[23147](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568639, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.254.131@KRBTEST.COM for oryx/127.11.254.188@KRBTEST.COM
Aug 12 21:04:00 dist-test-slave-vpvm krb5kdc[23147](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568639, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.254.131@KRBTEST.COM for oryx/127.11.254.190@KRBTEST.COM
Aug 12 21:04:00 dist-test-slave-vpvm krb5kdc[23147](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568639, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.254.131@KRBTEST.COM for oryx/127.11.254.189@KRBTEST.COM
I20260812 21:04:00.042213 24875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:00.031126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.131:35837 (local address 127.11.254.188:36857)
0812 21:04:00.031169 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0812 21:04:00.031171 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:04:00.031383 (+ 212us) server_negotiation.cc:407] Connection header received
0812 21:04:00.032482 (+ 1099us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:04:00.032484 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:04:00.032497 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:04:00.032527 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:04:00.033859 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:00.034647 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:00.035824 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:00.036095 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:00.036155 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:00.038863 (+ 2708us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:04:00.038865 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:04:00.038865 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:04:00.038873 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:04:00.040598 (+ 1725us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:04:00.041019 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:04:00.041020 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:04:00.041021 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:04:00.041082 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:04:00.041358 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:04:00.041359 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:04:00.041360 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:04:00.041488 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:04:00.041547 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0812 21:04:00.042033 (+ 486us) server_negotiation.cc:299] Negotiation successful
0812 21:04:00.042117 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 21:04:00.042730 25345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:00.031093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.188:36857 (local address 127.11.254.131:35837)
0812 21:04:00.031268 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0812 21:04:00.031305 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0812 21:04:00.032361 (+ 1056us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:04:00.032683 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:04:00.032687 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:04:00.033723 (+ 1036us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:04:00.033777 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:00.033779 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:00.035073 (+ 1294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:00.035073 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:00.035744 (+ 671us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:00.035745 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:00.036225 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:00.036226 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:00.036372 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:00.036926 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:04:00.036935 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:04:00.038751 (+ 1816us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:04:00.040659 (+ 1908us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:04:00.040667 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:04:00.040669 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:04:00.040946 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:04:00.041155 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:04:00.041155 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:04:00.041156 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:04:00.041287 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:04:00.041573 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:04:00.041580 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:04:00.041754 (+ 174us) client_negotiation.cc:769] Sending connection context
0812 21:04:00.041860 (+ 106us) client_negotiation.cc:240] Negotiation successful
0812 21:04:00.042479 (+ 619us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"mutex_wait_us":3,"thread_start_us":107,"threads_started":1}
I20260812 21:04:00.043404 25362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:00.033647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41328 (local address 127.11.254.189:36155)
0812 21:04:00.034128 (+ 481us) server_negotiation.cc:206] Beginning negotiation
0812 21:04:00.034130 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:04:00.034136 (+ 6us) server_negotiation.cc:407] Connection header received
0812 21:04:00.034156 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:04:00.034157 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:04:00.034167 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:04:00.034218 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:04:00.035009 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:00.035772 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:00.036812 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:00.037881 (+ 1069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:00.038247 (+ 366us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:00.039322 (+ 1075us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:04:00.039323 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:04:00.039323 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:04:00.039331 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:04:00.041661 (+ 2330us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:04:00.042102 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:04:00.042102 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:04:00.042103 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:04:00.042159 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:04:00.042466 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:04:00.042467 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:04:00.042467 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:04:00.042611 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:04:00.043182 (+ 571us) server_negotiation.cc:1035] Waiting for connection context
0812 21:04:00.043223 (+ 41us) server_negotiation.cc:299] Negotiation successful
0812 21:04:00.043309 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":464,"thread_start_us":73,"threads_started":1}
I20260812 21:04:00.043615 25343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:00.031093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:43007 (local address 127.11.254.131:56709)
0812 21:04:00.031262 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0812 21:04:00.031305 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 21:04:00.032352 (+ 1047us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:04:00.032683 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:04:00.032688 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:04:00.033306 (+ 618us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:04:00.033537 (+ 231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:00.033544 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:00.034874 (+ 1330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:00.034875 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:00.035880 (+ 1005us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:00.035883 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:00.037391 (+ 1508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:00.037392 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:00.037454 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:00.037977 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:04:00.037982 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:04:00.039986 (+ 2004us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:04:00.042159 (+ 2173us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:04:00.042160 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:04:00.042160 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:04:00.042364 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:04:00.042575 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:04:00.042576 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:04:00.042576 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:04:00.042638 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:04:00.043287 (+ 649us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:04:00.043288 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:04:00.043332 (+ 44us) client_negotiation.cc:769] Sending connection context
0812 21:04:00.043396 (+ 64us) client_negotiation.cc:240] Negotiation successful
0812 21:04:00.043506 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":98,"threads_started":1}
I20260812 21:04:00.044148 23283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:00.031107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.131:56709 (local address 127.11.254.190:43007)
0812 21:04:00.031162 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0812 21:04:00.031164 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:04:00.031429 (+ 265us) server_negotiation.cc:407] Connection header received
0812 21:04:00.032502 (+ 1073us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:04:00.032503 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:04:00.032534 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:04:00.032586 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:04:00.033970 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:00.034751 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:00.036982 (+ 2231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:00.037275 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:00.037312 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:00.040106 (+ 2794us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:04:00.040108 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:04:00.040108 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:04:00.040116 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:04:00.042055 (+ 1939us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:04:00.042442 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:04:00.042443 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:04:00.042443 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:04:00.042503 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:04:00.043040 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:04:00.043042 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:04:00.043042 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:04:00.043198 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:04:00.043256 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0812 21:04:00.043981 (+ 725us) server_negotiation.cc:299] Negotiation successful
0812 21:04:00.044059 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260812 21:04:00.044739 25338 heartbeater.cc:347] Connected to a master server at 127.11.254.188:36857
I20260812 21:04:00.044838 25338 heartbeater.cc:464] Registering TS with master...
I20260812 21:04:00.045037 25338 heartbeater.cc:511] Master 127.11.254.188:36857 requested a full tablet report, sending...
I20260812 21:04:00.044608 25339 heartbeater.cc:347] Connected to a master server at 127.11.254.190:43007
I20260812 21:04:00.045392 25339 heartbeater.cc:464] Registering TS with master...
I20260812 21:04:00.045490 25339 heartbeater.cc:511] Master 127.11.254.190:43007 requested a full tablet report, sending...
I20260812 21:04:00.045502 24582 ts_manager.cc:194] Registered new tserver with Master: 228262922b09413f8fe37401b0a932bd (127.11.254.131:46307)
I20260812 21:04:00.045832 23224 ts_manager.cc:194] Re-registered known tserver with Master: 228262922b09413f8fe37401b0a932bd (127.11.254.131:46307)
I20260812 21:04:00.046314 24345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:00.031549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.131:54037 (local address 127.11.254.189:36155)
0812 21:04:00.031594 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0812 21:04:00.031596 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:04:00.032579 (+ 983us) server_negotiation.cc:407] Connection header received
0812 21:04:00.032673 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:04:00.032674 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:04:00.032685 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:04:00.032711 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:04:00.033690 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:00.034509 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:00.037046 (+ 2537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:00.037324 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:00.037358 (+ 34us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:00.043401 (+ 6043us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:04:00.043402 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:04:00.043403 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:04:00.043409 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:04:00.044898 (+ 1489us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:04:00.045283 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:04:00.045285 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:04:00.045285 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:04:00.045339 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:04:00.045693 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:04:00.045694 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:04:00.045694 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:04:00.045821 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:04:00.045863 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0812 21:04:00.046149 (+ 286us) server_negotiation.cc:299] Negotiation successful
0812 21:04:00.046225 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 21:04:00.046344 23224 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.11.254.131@KRBTEST.COM'} at 127.11.254.131:56709
I20260812 21:04:00.046797 25346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:00.031396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.189:36155 (local address 127.11.254.131:54037)
0812 21:04:00.032523 (+ 1127us) negotiation.cc:107] Waiting for socket to connect
0812 21:04:00.032532 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 21:04:00.032605 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:04:00.032770 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:04:00.032771 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:04:00.033232 (+ 461us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:04:00.033537 (+ 305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:00.033545 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:00.034752 (+ 1207us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:00.034753 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:00.036963 (+ 2210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:00.036965 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:00.037436 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:00.037437 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:00.037491 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:00.038064 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:04:00.038068 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:04:00.043314 (+ 5246us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:04:00.044970 (+ 1656us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:04:00.044971 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:04:00.044972 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:04:00.045189 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:04:00.045479 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:04:00.045480 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:04:00.045480 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:04:00.045616 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:04:00.045982 (+ 366us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:04:00.045983 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:04:00.046027 (+ 44us) client_negotiation.cc:769] Sending connection context
0812 21:04:00.046582 (+ 555us) client_negotiation.cc:240] Negotiation successful
0812 21:04:00.046690 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1111,"mutex_wait_us":918,"thread_start_us":56,"threads_started":1}
I20260812 21:04:00.047286 25341 heartbeater.cc:347] Connected to a master server at 127.11.254.189:36155
I20260812 21:04:00.047340 25341 heartbeater.cc:464] Registering TS with master...
I20260812 21:04:00.047403 25341 heartbeater.cc:511] Master 127.11.254.189:36155 requested a full tablet report, sending...
I20260812 21:04:00.047683 24246 ts_manager.cc:194] Registered new tserver with Master: 228262922b09413f8fe37401b0a932bd (127.11.254.131:46307)
I20260812 21:04:00.047887 25365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:00.033770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32976 (local address 127.11.254.188:36857)
0812 21:04:00.037657 (+ 3887us) server_negotiation.cc:206] Beginning negotiation
0812 21:04:00.037659 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:04:00.037666 (+ 7us) server_negotiation.cc:407] Connection header received
0812 21:04:00.037686 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:04:00.037687 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:04:00.037697 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:04:00.038810 (+ 1113us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:04:00.038851 (+ 41us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:00.039567 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:00.040516 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:00.040824 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:00.040990 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:00.043909 (+ 2919us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:04:00.043910 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:04:00.043911 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:04:00.043929 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:04:00.046551 (+ 2622us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:04:00.046908 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:04:00.046909 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:04:00.046909 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:04:00.046989 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:04:00.047195 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:04:00.047195 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:04:00.047196 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:04:00.047336 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:04:00.047676 (+ 340us) server_negotiation.cc:1035] Waiting for connection context
0812 21:04:00.047719 (+ 43us) server_negotiation.cc:299] Negotiation successful
0812 21:04:00.047797 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3860,"thread_start_us":62,"threads_started":1}
I20260812 21:04:00.048823 25357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:00.033298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38766 (local address 127.11.254.190:43007)
0812 21:04:00.035875 (+ 2577us) server_negotiation.cc:206] Beginning negotiation
0812 21:04:00.035877 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:04:00.035884 (+ 7us) server_negotiation.cc:407] Connection header received
0812 21:04:00.035902 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:04:00.035903 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:04:00.035914 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:04:00.035969 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:04:00.036668 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:00.037379 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:00.042855 (+ 5476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:00.044409 (+ 1554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:00.044466 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:00.045842 (+ 1376us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:04:00.045843 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:04:00.045843 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:04:00.045850 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:04:00.047763 (+ 1913us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:04:00.048127 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:04:00.048129 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:04:00.048129 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:04:00.048189 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:04:00.048357 (+ 168us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:04:00.048358 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:04:00.048358 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:04:00.048465 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:04:00.048550 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0812 21:04:00.048654 (+ 104us) server_negotiation.cc:299] Negotiation successful
0812 21:04:00.048734 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2545,"thread_start_us":96,"threads_started":1}
I20260812 21:04:00.050464 23224 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38766:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 21:04:00.052019 23224 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 21:04:00.059398 24173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:00.055496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.130:39443 (local address 127.0.0.1:57258)
0812 21:04:00.055718 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0812 21:04:00.055729 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 21:04:00.055771 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:04:00.056012 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:04:00.056013 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:04:00.056015 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:04:00.056065 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:00.056067 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:00.057050 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:00.057051 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:00.058065 (+ 1014us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:00.058068 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:00.059153 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:00.059154 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:00.059209 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:00.059213 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 21:04:00.059256 (+ 43us) client_negotiation.cc:240] Negotiation successful
0812 21:04:00.059302 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":204}
I20260812 21:04:00.059405 25383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:00.055648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57258 (local address 127.11.254.130:39443)
0812 21:04:00.055830 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0812 21:04:00.055835 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 21:04:00.055848 (+ 13us) server_negotiation.cc:407] Connection header received
0812 21:04:00.055891 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:04:00.055893 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:04:00.055910 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:04:00.056139 (+ 229us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:04:00.056207 (+ 68us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:00.056964 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:00.058191 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:00.059016 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:00.059171 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:00.059249 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0812 21:04:00.059278 (+ 29us) server_negotiation.cc:299] Negotiation successful
0812 21:04:00.059317 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":95,"threads_started":1}
I20260812 21:04:00.060037 25380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:00.055567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.131:46307 (local address 127.0.0.1:41230)
0812 21:04:00.055841 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0812 21:04:00.055850 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 21:04:00.055890 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:04:00.056161 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:04:00.056162 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:04:00.056164 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:04:00.056199 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:00.056201 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:00.057444 (+ 1243us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:00.057444 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:00.058634 (+ 1190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:00.058637 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:00.059807 (+ 1170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:00.059808 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:00.059858 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:00.059862 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 21:04:00.059924 (+ 62us) client_negotiation.cc:240] Negotiation successful
0812 21:04:00.059969 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":136,"threads_started":1}
I20260812 21:04:00.060071 25382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:00.055648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41230 (local address 127.11.254.131:46307)
0812 21:04:00.056004 (+ 356us) server_negotiation.cc:206] Beginning negotiation
0812 21:04:00.056008 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 21:04:00.056017 (+ 9us) server_negotiation.cc:407] Connection header received
0812 21:04:00.056056 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:04:00.056058 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:04:00.056074 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:04:00.056158 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:04:00.056264 (+ 106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:00.057351 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:00.058715 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:00.059454 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:00.059656 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:00.059716 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0812 21:04:00.059959 (+ 243us) server_negotiation.cc:299] Negotiation successful
0812 21:04:00.060000 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":317,"thread_start_us":72,"threads_started":1}
I20260812 21:04:00.060223 25000 tablet_service.cc:1467] Processing CreateTablet for tablet 4f9ac0eb8a14441b8294545eb00d1c49 (DEFAULT_TABLE table=test-table [id=f168ad8852f34734a00e9437fc58cd31]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:04:00.060724 25000 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4f9ac0eb8a14441b8294545eb00d1c49. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:04:00.061283 25376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:00.055569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.129:35727 (local address 127.0.0.1:49082)
0812 21:04:00.056129 (+ 560us) negotiation.cc:107] Waiting for socket to connect
0812 21:04:00.056138 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 21:04:00.056178 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:04:00.056475 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:04:00.056476 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:04:00.056478 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:04:00.056523 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:00.056524 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:00.057714 (+ 1190us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:00.057716 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:00.059020 (+ 1304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:00.059023 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:00.061066 (+ 2043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:00.061067 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:00.061129 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:00.061133 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 21:04:00.061178 (+ 45us) client_negotiation.cc:240] Negotiation successful
0812 21:04:00.061218 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":544,"thread_start_us":105,"threads_started":1}
I20260812 21:04:00.061205 25235 tablet_service.cc:1467] Processing CreateTablet for tablet 4f9ac0eb8a14441b8294545eb00d1c49 (DEFAULT_TABLE table=test-table [id=f168ad8852f34734a00e9437fc58cd31]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:04:00.061558 25235 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4f9ac0eb8a14441b8294545eb00d1c49. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:04:00.062000 25384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:00.055932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49082 (local address 127.11.254.129:35727)
0812 21:04:00.056281 (+ 349us) server_negotiation.cc:206] Beginning negotiation
0812 21:04:00.056285 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 21:04:00.056294 (+ 9us) server_negotiation.cc:407] Connection header received
0812 21:04:00.056331 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:04:00.056333 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:04:00.056348 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:04:00.056412 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:04:00.056596 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:00.057591 (+ 995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:00.059104 (+ 1513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:00.060913 (+ 1809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:00.060981 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:00.061031 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0812 21:04:00.061650 (+ 619us) server_negotiation.cc:299] Negotiation successful
0812 21:04:00.061698 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":315,"thread_start_us":88,"threads_started":1}
I20260812 21:04:00.062958 24760 tablet_service.cc:1467] Processing CreateTablet for tablet 4f9ac0eb8a14441b8294545eb00d1c49 (DEFAULT_TABLE table=test-table [id=f168ad8852f34734a00e9437fc58cd31]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:04:00.063315 24760 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4f9ac0eb8a14441b8294545eb00d1c49. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:04:00.063861 24173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:00.060167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.188:36857 (local address 127.0.0.1:32980)
0812 21:04:00.060209 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0812 21:04:00.060220 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 21:04:00.060274 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:04:00.060657 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:04:00.060658 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:04:00.060660 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:04:00.060700 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:00.060702 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:00.061567 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:00.061568 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:00.062637 (+ 1069us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:00.062640 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:00.063613 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:00.063614 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:00.063673 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:00.063677 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 21:04:00.063732 (+ 55us) client_negotiation.cc:240] Negotiation successful
0812 21:04:00.063774 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25}
I20260812 21:04:00.064181 25365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:00.060338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32980 (local address 127.11.254.188:36857)
0812 21:04:00.060387 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0812 21:04:00.060389 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:04:00.060397 (+ 8us) server_negotiation.cc:407] Connection header received
0812 21:04:00.060429 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:04:00.060430 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:04:00.060454 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:04:00.060522 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:04:00.060794 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:00.061456 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:00.062796 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:00.063424 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:00.063505 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:00.063573 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0812 21:04:00.064058 (+ 485us) server_negotiation.cc:299] Negotiation successful
0812 21:04:00.064113 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260812 21:04:00.064508 24597 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 7e4700f28485420fbba29f2e545c71df [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.9] Dedup: 1.7->[1.8-1.9]
I20260812 21:04:00.065521 24643 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e4700f28485420fbba29f2e545c71df [sys.catalog]: SysCatalogTable state changed. Reason: New leader c90e0f61a5354abbbf96692370a65ebd. Latest consensus state: current_term: 1 leader_uuid: "c90e0f61a5354abbbf96692370a65ebd" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c90e0f61a5354abbbf96692370a65ebd" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 43007 } } peers { permanent_uuid: "5026b3627f2e49f8b4e7cf1521420cd1" member_type: VOTER last_known_addr { host: "127.11.254.189" port: 36155 } attrs { promote: false } } peers { permanent_uuid: "7e4700f28485420fbba29f2e545c71df" member_type: NON_VOTER last_known_addr { host: "127.11.254.188" port: 36857 } attrs { promote: true } } }
I20260812 21:04:00.065632 24643 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e4700f28485420fbba29f2e545c71df [sys.catalog]: This master's current role is: LEARNER
I20260812 21:04:00.067337 25391 tablet_bootstrap.cc:492] T 4f9ac0eb8a14441b8294545eb00d1c49 P 228262922b09413f8fe37401b0a932bd: Bootstrap starting.
I20260812 21:04:00.068274 25396 tablet_bootstrap.cc:492] T 4f9ac0eb8a14441b8294545eb00d1c49 P 8e16af0efe6c4f698e93d11250de344b: Bootstrap starting.
I20260812 21:04:00.068352 25391 tablet_bootstrap.cc:654] T 4f9ac0eb8a14441b8294545eb00d1c49 P 228262922b09413f8fe37401b0a932bd: Neither blocks nor log segments found. Creating new log.
I20260812 21:04:00.068766 25389 tablet_bootstrap.cc:492] T 4f9ac0eb8a14441b8294545eb00d1c49 P a9da4e76358f4ee18755f38b4d8de7ad: Bootstrap starting.
I20260812 21:04:00.069361 25391 log.cc:824] T 4f9ac0eb8a14441b8294545eb00d1c49 P 228262922b09413f8fe37401b0a932bd: Log is configured to *not* fsync() on all Append() calls
I20260812 21:04:00.069496 25389 tablet_bootstrap.cc:654] T 4f9ac0eb8a14441b8294545eb00d1c49 P a9da4e76358f4ee18755f38b4d8de7ad: Neither blocks nor log segments found. Creating new log.
I20260812 21:04:00.069710 25396 tablet_bootstrap.cc:654] T 4f9ac0eb8a14441b8294545eb00d1c49 P 8e16af0efe6c4f698e93d11250de344b: Neither blocks nor log segments found. Creating new log.
I20260812 21:04:00.070329 25391 tablet_bootstrap.cc:492] T 4f9ac0eb8a14441b8294545eb00d1c49 P 228262922b09413f8fe37401b0a932bd: No bootstrap required, opened a new log
I20260812 21:04:00.070415 25391 ts_tablet_manager.cc:1397] T 4f9ac0eb8a14441b8294545eb00d1c49 P 228262922b09413f8fe37401b0a932bd: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 21:04:00.070408 25389 log.cc:824] T 4f9ac0eb8a14441b8294545eb00d1c49 P a9da4e76358f4ee18755f38b4d8de7ad: Log is configured to *not* fsync() on all Append() calls
I20260812 21:04:00.070680 25396 log.cc:824] T 4f9ac0eb8a14441b8294545eb00d1c49 P 8e16af0efe6c4f698e93d11250de344b: Log is configured to *not* fsync() on all Append() calls
I20260812 21:04:00.071295 25389 tablet_bootstrap.cc:492] T 4f9ac0eb8a14441b8294545eb00d1c49 P a9da4e76358f4ee18755f38b4d8de7ad: No bootstrap required, opened a new log
I20260812 21:04:00.071403 25389 ts_tablet_manager.cc:1397] T 4f9ac0eb8a14441b8294545eb00d1c49 P a9da4e76358f4ee18755f38b4d8de7ad: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 21:04:00.071655 25396 tablet_bootstrap.cc:492] T 4f9ac0eb8a14441b8294545eb00d1c49 P 8e16af0efe6c4f698e93d11250de344b: No bootstrap required, opened a new log
I20260812 21:04:00.071719 25396 ts_tablet_manager.cc:1397] T 4f9ac0eb8a14441b8294545eb00d1c49 P 8e16af0efe6c4f698e93d11250de344b: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20260812 21:04:00.074124 25391 raft_consensus.cc:348] T 4f9ac0eb8a14441b8294545eb00d1c49 P 228262922b09413f8fe37401b0a932bd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9da4e76358f4ee18755f38b4d8de7ad" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 39443 } } peers { permanent_uuid: "228262922b09413f8fe37401b0a932bd" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 46307 } } peers { permanent_uuid: "8e16af0efe6c4f698e93d11250de344b" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 35727 } }
I20260812 21:04:00.074297 25391 raft_consensus.cc:374] T 4f9ac0eb8a14441b8294545eb00d1c49 P 228262922b09413f8fe37401b0a932bd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:04:00.074353 25391 raft_consensus.cc:729] T 4f9ac0eb8a14441b8294545eb00d1c49 P 228262922b09413f8fe37401b0a932bd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 228262922b09413f8fe37401b0a932bd, State: Initialized, Role: FOLLOWER
I20260812 21:04:00.074534 25391 consensus_queue.cc:260] T 4f9ac0eb8a14441b8294545eb00d1c49 P 228262922b09413f8fe37401b0a932bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9da4e76358f4ee18755f38b4d8de7ad" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 39443 } } peers { permanent_uuid: "228262922b09413f8fe37401b0a932bd" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 46307 } } peers { permanent_uuid: "8e16af0efe6c4f698e93d11250de344b" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 35727 } }
I20260812 21:04:00.074821 25391 ts_tablet_manager.cc:1428] T 4f9ac0eb8a14441b8294545eb00d1c49 P 228262922b09413f8fe37401b0a932bd: Time spent starting tablet: real 0.004s user 0.000s sys 0.005s
I20260812 21:04:00.074920 25339 heartbeater.cc:503] Master 127.11.254.190:43007 was elected leader, sending a full tablet report...
I20260812 21:04:00.075282 25396 raft_consensus.cc:348] T 4f9ac0eb8a14441b8294545eb00d1c49 P 8e16af0efe6c4f698e93d11250de344b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9da4e76358f4ee18755f38b4d8de7ad" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 39443 } } peers { permanent_uuid: "228262922b09413f8fe37401b0a932bd" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 46307 } } peers { permanent_uuid: "8e16af0efe6c4f698e93d11250de344b" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 35727 } }
I20260812 21:04:00.075464 25396 raft_consensus.cc:374] T 4f9ac0eb8a14441b8294545eb00d1c49 P 8e16af0efe6c4f698e93d11250de344b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:04:00.075567 25396 raft_consensus.cc:729] T 4f9ac0eb8a14441b8294545eb00d1c49 P 8e16af0efe6c4f698e93d11250de344b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e16af0efe6c4f698e93d11250de344b, State: Initialized, Role: FOLLOWER
I20260812 21:04:00.075717 25396 consensus_queue.cc:260] T 4f9ac0eb8a14441b8294545eb00d1c49 P 8e16af0efe6c4f698e93d11250de344b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9da4e76358f4ee18755f38b4d8de7ad" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 39443 } } peers { permanent_uuid: "228262922b09413f8fe37401b0a932bd" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 46307 } } peers { permanent_uuid: "8e16af0efe6c4f698e93d11250de344b" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 35727 } }
I20260812 21:04:00.076076 25396 ts_tablet_manager.cc:1428] T 4f9ac0eb8a14441b8294545eb00d1c49 P 8e16af0efe6c4f698e93d11250de344b: Time spent starting tablet: real 0.004s user 0.004s sys 0.001s
I20260812 21:04:00.076429 24867 heartbeater.cc:503] Master 127.11.254.190:43007 was elected leader, sending a full tablet report...
I20260812 21:04:00.077338 25389 raft_consensus.cc:348] T 4f9ac0eb8a14441b8294545eb00d1c49 P a9da4e76358f4ee18755f38b4d8de7ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9da4e76358f4ee18755f38b4d8de7ad" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 39443 } } peers { permanent_uuid: "228262922b09413f8fe37401b0a932bd" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 46307 } } peers { permanent_uuid: "8e16af0efe6c4f698e93d11250de344b" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 35727 } }
I20260812 21:04:00.077560 25389 raft_consensus.cc:374] T 4f9ac0eb8a14441b8294545eb00d1c49 P a9da4e76358f4ee18755f38b4d8de7ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:04:00.077621 25389 raft_consensus.cc:729] T 4f9ac0eb8a14441b8294545eb00d1c49 P a9da4e76358f4ee18755f38b4d8de7ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9da4e76358f4ee18755f38b4d8de7ad, State: Initialized, Role: FOLLOWER
I20260812 21:04:00.077731 25389 consensus_queue.cc:260] T 4f9ac0eb8a14441b8294545eb00d1c49 P a9da4e76358f4ee18755f38b4d8de7ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9da4e76358f4ee18755f38b4d8de7ad" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 39443 } } peers { permanent_uuid: "228262922b09413f8fe37401b0a932bd" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 46307 } } peers { permanent_uuid: "8e16af0efe6c4f698e93d11250de344b" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 35727 } }
I20260812 21:04:00.078032 25389 ts_tablet_manager.cc:1428] T 4f9ac0eb8a14441b8294545eb00d1c49 P a9da4e76358f4ee18755f38b4d8de7ad: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
I20260812 21:04:00.078096 25114 heartbeater.cc:503] Master 127.11.254.190:43007 was elected leader, sending a full tablet report...
W20260812 21:04:00.113042 25116 tablet.cc:2369] T 4f9ac0eb8a14441b8294545eb00d1c49 P a9da4e76358f4ee18755f38b4d8de7ad: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 21:04:00.208595 24872 tablet.cc:2369] T 4f9ac0eb8a14441b8294545eb00d1c49 P 8e16af0efe6c4f698e93d11250de344b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 21:04:00.281260 25342 tablet.cc:2369] T 4f9ac0eb8a14441b8294545eb00d1c49 P 228262922b09413f8fe37401b0a932bd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 21:04:00.419787 25408 raft_consensus.cc:482] T 4f9ac0eb8a14441b8294545eb00d1c49 P 8e16af0efe6c4f698e93d11250de344b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 21:04:00.419946 25408 raft_consensus.cc:504] T 4f9ac0eb8a14441b8294545eb00d1c49 P 8e16af0efe6c4f698e93d11250de344b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9da4e76358f4ee18755f38b4d8de7ad" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 39443 } } peers { permanent_uuid: "228262922b09413f8fe37401b0a932bd" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 46307 } } peers { permanent_uuid: "8e16af0efe6c4f698e93d11250de344b" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 35727 } }
I20260812 21:04:00.420296 25408 leader_election.cc:288] T 4f9ac0eb8a14441b8294545eb00d1c49 P 8e16af0efe6c4f698e93d11250de344b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a9da4e76358f4ee18755f38b4d8de7ad (127.11.254.130:39443), 228262922b09413f8fe37401b0a932bd (127.11.254.131:46307)
I20260812 21:04:00.423259 25382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:00.420576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.129:43189 (local address 127.11.254.131:46307)
0812 21:04:00.420627 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0812 21:04:00.420629 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:04:00.420710 (+ 81us) server_negotiation.cc:407] Connection header received
0812 21:04:00.420790 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:04:00.420791 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:04:00.420803 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:04:00.420853 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:04:00.421032 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:00.421538 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:00.422538 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:00.422919 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:00.422986 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:00.422994 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 21:04:00.423139 (+ 145us) server_negotiation.cc:299] Negotiation successful
0812 21:04:00.423184 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260812 21:04:00.423280 25410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:00.420479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.131:46307 (local address 127.11.254.129:43189)
0812 21:04:00.420632 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0812 21:04:00.420644 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 21:04:00.420721 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:04:00.420865 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:04:00.420866 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:04:00.420868 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:04:00.420952 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:00.420955 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:00.421620 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:00.421621 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:00.422459 (+ 838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:00.422461 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:00.422995 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:00.422996 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:00.423065 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:00.423068 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 21:04:00.423109 (+ 41us) client_negotiation.cc:240] Negotiation successful
0812 21:04:00.423153 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":69,"threads_started":1}
I20260812 21:04:00.423360 25383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:00.420653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.129:38689 (local address 127.11.254.130:39443)
0812 21:04:00.420724 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0812 21:04:00.420725 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 21:04:00.420735 (+ 10us) server_negotiation.cc:407] Connection header received
0812 21:04:00.420810 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:04:00.420811 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:04:00.420819 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:04:00.420859 (+ 40us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:04:00.421137 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:00.421622 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:00.422570 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:00.422909 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:00.422967 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:00.422976 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 21:04:00.423258 (+ 282us) server_negotiation.cc:299] Negotiation successful
0812 21:04:00.423302 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 21:04:00.423623 25411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:00.420534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.130:39443 (local address 127.11.254.129:38689)
0812 21:04:00.420672 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0812 21:04:00.420681 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 21:04:00.420737 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:04:00.420870 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:04:00.420871 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:04:00.420873 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:04:00.421054 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:00.421056 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:00.421706 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:00.421706 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:00.422486 (+ 780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:00.422489 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:00.422980 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:00.422980 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:00.423175 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:00.423180 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 21:04:00.423239 (+ 59us) client_negotiation.cc:240] Negotiation successful
0812 21:04:00.423279 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":62,"threads_started":1}
I20260812 21:04:00.423736 25261 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f9ac0eb8a14441b8294545eb00d1c49" candidate_uuid: "8e16af0efe6c4f698e93d11250de344b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "228262922b09413f8fe37401b0a932bd" is_pre_election: true
I20260812 21:04:00.423954 25261 raft_consensus.cc:2454] T 4f9ac0eb8a14441b8294545eb00d1c49 P 228262922b09413f8fe37401b0a932bd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e16af0efe6c4f698e93d11250de344b in term 0.
I20260812 21:04:00.423956 25036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f9ac0eb8a14441b8294545eb00d1c49" candidate_uuid: "8e16af0efe6c4f698e93d11250de344b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9da4e76358f4ee18755f38b4d8de7ad" is_pre_election: true
I20260812 21:04:00.424104 25036 raft_consensus.cc:2454] T 4f9ac0eb8a14441b8294545eb00d1c49 P a9da4e76358f4ee18755f38b4d8de7ad [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e16af0efe6c4f698e93d11250de344b in term 0.
I20260812 21:04:00.424233 24692 leader_election.cc:302] T 4f9ac0eb8a14441b8294545eb00d1c49 P 8e16af0efe6c4f698e93d11250de344b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 228262922b09413f8fe37401b0a932bd, 8e16af0efe6c4f698e93d11250de344b; no voters:
I20260812 21:04:00.424388 25408 raft_consensus.cc:2781] T 4f9ac0eb8a14441b8294545eb00d1c49 P 8e16af0efe6c4f698e93d11250de344b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 21:04:00.424453 25408 raft_consensus.cc:482] T 4f9ac0eb8a14441b8294545eb00d1c49 P 8e16af0efe6c4f698e93d11250de344b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 21:04:00.424475 25408 raft_consensus.cc:3037] T 4f9ac0eb8a14441b8294545eb00d1c49 P 8e16af0efe6c4f698e93d11250de344b [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:04:00.425284 25408 raft_consensus.cc:504] T 4f9ac0eb8a14441b8294545eb00d1c49 P 8e16af0efe6c4f698e93d11250de344b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9da4e76358f4ee18755f38b4d8de7ad" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 39443 } } peers { permanent_uuid: "228262922b09413f8fe37401b0a932bd" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 46307 } } peers { permanent_uuid: "8e16af0efe6c4f698e93d11250de344b" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 35727 } }
I20260812 21:04:00.425421 25408 leader_election.cc:288] T 4f9ac0eb8a14441b8294545eb00d1c49 P 8e16af0efe6c4f698e93d11250de344b [CANDIDATE]: Term 1 election: Requested vote from peers a9da4e76358f4ee18755f38b4d8de7ad (127.11.254.130:39443), 228262922b09413f8fe37401b0a932bd (127.11.254.131:46307)
I20260812 21:04:00.425570 25036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f9ac0eb8a14441b8294545eb00d1c49" candidate_uuid: "8e16af0efe6c4f698e93d11250de344b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9da4e76358f4ee18755f38b4d8de7ad"
I20260812 21:04:00.425643 25036 raft_consensus.cc:3037] T 4f9ac0eb8a14441b8294545eb00d1c49 P a9da4e76358f4ee18755f38b4d8de7ad [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:04:00.425730 25261 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f9ac0eb8a14441b8294545eb00d1c49" candidate_uuid: "8e16af0efe6c4f698e93d11250de344b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "228262922b09413f8fe37401b0a932bd"
I20260812 21:04:00.425818 25261 raft_consensus.cc:3037] T 4f9ac0eb8a14441b8294545eb00d1c49 P 228262922b09413f8fe37401b0a932bd [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:04:00.426614 25036 raft_consensus.cc:2454] T 4f9ac0eb8a14441b8294545eb00d1c49 P a9da4e76358f4ee18755f38b4d8de7ad [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e16af0efe6c4f698e93d11250de344b in term 1.
I20260812 21:04:00.426818 24689 leader_election.cc:302] T 4f9ac0eb8a14441b8294545eb00d1c49 P 8e16af0efe6c4f698e93d11250de344b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e16af0efe6c4f698e93d11250de344b, a9da4e76358f4ee18755f38b4d8de7ad; no voters:
I20260812 21:04:00.426959 25408 raft_consensus.cc:2781] T 4f9ac0eb8a14441b8294545eb00d1c49 P 8e16af0efe6c4f698e93d11250de344b [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:04:00.427124 25408 raft_consensus.cc:686] T 4f9ac0eb8a14441b8294545eb00d1c49 P 8e16af0efe6c4f698e93d11250de344b [term 1 LEADER]: Becoming Leader. State: Replica: 8e16af0efe6c4f698e93d11250de344b, State: Running, Role: LEADER
I20260812 21:04:00.427304 25408 consensus_queue.cc:237] T 4f9ac0eb8a14441b8294545eb00d1c49 P 8e16af0efe6c4f698e93d11250de344b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9da4e76358f4ee18755f38b4d8de7ad" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 39443 } } peers { permanent_uuid: "228262922b09413f8fe37401b0a932bd" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 46307 } } peers { permanent_uuid: "8e16af0efe6c4f698e93d11250de344b" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 35727 } }
I20260812 21:04:00.427357 25261 raft_consensus.cc:2454] T 4f9ac0eb8a14441b8294545eb00d1c49 P 228262922b09413f8fe37401b0a932bd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e16af0efe6c4f698e93d11250de344b in term 1.
I20260812 21:04:00.428239 23224 catalog_manager.cc:5495] T 4f9ac0eb8a14441b8294545eb00d1c49 P 8e16af0efe6c4f698e93d11250de344b reported cstate change: term changed from 0 to 1, leader changed from <none> to 8e16af0efe6c4f698e93d11250de344b (127.11.254.129). New cstate: current_term: 1 leader_uuid: "8e16af0efe6c4f698e93d11250de344b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9da4e76358f4ee18755f38b4d8de7ad" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 39443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "228262922b09413f8fe37401b0a932bd" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 46307 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8e16af0efe6c4f698e93d11250de344b" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 35727 } health_report { overall_health: HEALTHY } } }
I20260812 21:04:00.428803 24174 sys_catalog.cc:455] T 00000000000000000000000000000000 P c90e0f61a5354abbbf96692370a65ebd [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c90e0f61a5354abbbf96692370a65ebd" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c90e0f61a5354abbbf96692370a65ebd" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 43007 } } peers { permanent_uuid: "5026b3627f2e49f8b4e7cf1521420cd1" member_type: VOTER last_known_addr { host: "127.11.254.189" port: 36155 } attrs { promote: false } } peers { permanent_uuid: "7e4700f28485420fbba29f2e545c71df" member_type: NON_VOTER last_known_addr { host: "127.11.254.188" port: 36857 } attrs { promote: true } } }
I20260812 21:04:00.428903 24174 sys_catalog.cc:458] T 00000000000000000000000000000000 P c90e0f61a5354abbbf96692370a65ebd [sys.catalog]: This master's current role is: LEADER
I20260812 21:04:00.430142 24174 raft_consensus.cc:1050] T 00000000000000000000000000000000 P c90e0f61a5354abbbf96692370a65ebd: attempting to promote NON_VOTER 7e4700f28485420fbba29f2e545c71df to VOTER
I20260812 21:04:00.430164 25416 raft_consensus.cc:1050] T 00000000000000000000000000000000 P c90e0f61a5354abbbf96692370a65ebd: attempting to promote NON_VOTER 7e4700f28485420fbba29f2e545c71df to VOTER
I20260812 21:04:00.430284 24174 consensus_queue.cc:237] T 00000000000000000000000000000000 P c90e0f61a5354abbbf96692370a65ebd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 1.10, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "c90e0f61a5354abbbf96692370a65ebd" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 43007 } } peers { permanent_uuid: "5026b3627f2e49f8b4e7cf1521420cd1" member_type: VOTER last_known_addr { host: "127.11.254.189" port: 36155 } attrs { promote: false } } peers { permanent_uuid: "7e4700f28485420fbba29f2e545c71df" member_type: VOTER last_known_addr { host: "127.11.254.188" port: 36857 } attrs { promote: false } }
W20260812 21:04:00.430500 25416 raft_consensus.cc:1053] T 00000000000000000000000000000000 P c90e0f61a5354abbbf96692370a65ebd: Unable to promote non-voter 7e4700f28485420fbba29f2e545c71df: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c90e0f61a5354abbbf96692370a65ebd" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 43007 } } peers { permanent_uuid: "5026b3627f2e49f8b4e7cf1521420cd1" member_type: VOTER last_known_addr { host: "127.11.254.189" port: 36155 } attrs { promote: false } } peers { permanent_uuid: "7e4700f28485420fbba29f2e545c71df" member_type: NON_VOTER last_known_addr { host: "127.11.254.188" port: 36857 } attrs { promote: true } }.
Pending config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "c90e0f61a5354abbbf96692370a65ebd" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 43007 } } peers { permanent_uuid: "5026b3627f2e49f8b4e7cf1521420cd1" member_type: VOTER last_known_addr { host: "127.11.254.189" port: 36155 } attrs { promote: false } } peers { permanent_uuid: "7e4700f28485420fbba29f2e545c71df" member_type: VOTER last_known_addr { host: "127.11.254.188" port: 36857 } attrs { promote: false } }
I20260812 21:04:00.430680 24262 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5026b3627f2e49f8b4e7cf1521420cd1 [term 1 FOLLOWER]: Refusing update from remote peer c90e0f61a5354abbbf96692370a65ebd: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20260812 21:04:00.430693 24597 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7e4700f28485420fbba29f2e545c71df [term 1 LEARNER]: Refusing update from remote peer c90e0f61a5354abbbf96692370a65ebd: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20260812 21:04:00.430891 25416 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c90e0f61a5354abbbf96692370a65ebd [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e4700f28485420fbba29f2e545c71df" member_type: VOTER last_known_addr { host: "127.11.254.188" port: 36857 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260812 21:04:00.431020 24175 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c90e0f61a5354abbbf96692370a65ebd [LEADER]: Connected to new peer: Peer: permanent_uuid: "5026b3627f2e49f8b4e7cf1521420cd1" member_type: VOTER last_known_addr { host: "127.11.254.189" port: 36155 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260812 21:04:00.431509 25416 raft_consensus.cc:2932] T 00000000000000000000000000000000 P c90e0f61a5354abbbf96692370a65ebd [term 1 LEADER]: Committing config change with OpId 1.11: config changed from index 7 to 11, 7e4700f28485420fbba29f2e545c71df (127.11.254.188) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "c90e0f61a5354abbbf96692370a65ebd" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 43007 } } peers { permanent_uuid: "5026b3627f2e49f8b4e7cf1521420cd1" member_type: VOTER last_known_addr { host: "127.11.254.189" port: 36155 } attrs { promote: false } } peers { permanent_uuid: "7e4700f28485420fbba29f2e545c71df" member_type: VOTER last_known_addr { host: "127.11.254.188" port: 36857 } attrs { promote: false } } }
I20260812 21:04:00.431643 24597 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 7e4700f28485420fbba29f2e545c71df [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 7 to 11, 7e4700f28485420fbba29f2e545c71df (127.11.254.188) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "c90e0f61a5354abbbf96692370a65ebd" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 43007 } } peers { permanent_uuid: "5026b3627f2e49f8b4e7cf1521420cd1" member_type: VOTER last_known_addr { host: "127.11.254.189" port: 36155 } attrs { promote: false } } peers { permanent_uuid: "7e4700f28485420fbba29f2e545c71df" member_type: VOTER last_known_addr { host: "127.11.254.188" port: 36857 } attrs { promote: false } } }
I20260812 21:04:00.431818 24262 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 5026b3627f2e49f8b4e7cf1521420cd1 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 7 to 11, 7e4700f28485420fbba29f2e545c71df (127.11.254.188) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "c90e0f61a5354abbbf96692370a65ebd" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 43007 } } peers { permanent_uuid: "5026b3627f2e49f8b4e7cf1521420cd1" member_type: VOTER last_known_addr { host: "127.11.254.189" port: 36155 } attrs { promote: false } } peers { permanent_uuid: "7e4700f28485420fbba29f2e545c71df" member_type: VOTER last_known_addr { host: "127.11.254.188" port: 36857 } attrs { promote: false } } }
I20260812 21:04:00.432847 24175 sys_catalog.cc:455] T 00000000000000000000000000000000 P c90e0f61a5354abbbf96692370a65ebd [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "c90e0f61a5354abbbf96692370a65ebd" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "c90e0f61a5354abbbf96692370a65ebd" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 43007 } } peers { permanent_uuid: "5026b3627f2e49f8b4e7cf1521420cd1" member_type: VOTER last_known_addr { host: "127.11.254.189" port: 36155 } attrs { promote: false } } peers { permanent_uuid: "7e4700f28485420fbba29f2e545c71df" member_type: VOTER last_known_addr { host: "127.11.254.188" port: 36857 } attrs { promote: false } } }
I20260812 21:04:00.432847 24643 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e4700f28485420fbba29f2e545c71df [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c90e0f61a5354abbbf96692370a65ebd" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "c90e0f61a5354abbbf96692370a65ebd" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 43007 } } peers { permanent_uuid: "5026b3627f2e49f8b4e7cf1521420cd1" member_type: VOTER last_known_addr { host: "127.11.254.189" port: 36155 } attrs { promote: false } } peers { permanent_uuid: "7e4700f28485420fbba29f2e545c71df" member_type: VOTER last_known_addr { host: "127.11.254.188" port: 36857 } attrs { promote: false } } }
I20260812 21:04:00.432933 24643 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e4700f28485420fbba29f2e545c71df [sys.catalog]: This master's current role is: FOLLOWER
I20260812 21:04:00.432933 24175 sys_catalog.cc:458] T 00000000000000000000000000000000 P c90e0f61a5354abbbf96692370a65ebd [sys.catalog]: This master's current role is: LEADER
I20260812 21:04:00.432974 25419 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5026b3627f2e49f8b4e7cf1521420cd1 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c90e0f61a5354abbbf96692370a65ebd" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "c90e0f61a5354abbbf96692370a65ebd" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 43007 } } peers { permanent_uuid: "5026b3627f2e49f8b4e7cf1521420cd1" member_type: VOTER last_known_addr { host: "127.11.254.189" port: 36155 } attrs { promote: false } } peers { permanent_uuid: "7e4700f28485420fbba29f2e545c71df" member_type: VOTER last_known_addr { host: "127.11.254.188" port: 36857 } attrs { promote: false } } }
I20260812 21:04:00.433048 25419 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5026b3627f2e49f8b4e7cf1521420cd1 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 21:04:00.433022 24175 sys_catalog.cc:455] T 00000000000000000000000000000000 P c90e0f61a5354abbbf96692370a65ebd [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c90e0f61a5354abbbf96692370a65ebd" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "c90e0f61a5354abbbf96692370a65ebd" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 43007 } } peers { permanent_uuid: "5026b3627f2e49f8b4e7cf1521420cd1" member_type: VOTER last_known_addr { host: "127.11.254.189" port: 36155 } attrs { promote: false } } peers { permanent_uuid: "7e4700f28485420fbba29f2e545c71df" member_type: VOTER last_known_addr { host: "127.11.254.188" port: 36857 } attrs { promote: false } } }
I20260812 21:04:00.433087 24175 sys_catalog.cc:458] T 00000000000000000000000000000000 P c90e0f61a5354abbbf96692370a65ebd [sys.catalog]: This master's current role is: LEADER
I20260812 21:04:00.433198 25416 sys_catalog.cc:455] T 00000000000000000000000000000000 P c90e0f61a5354abbbf96692370a65ebd [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c90e0f61a5354abbbf96692370a65ebd" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "c90e0f61a5354abbbf96692370a65ebd" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 43007 } } peers { permanent_uuid: "5026b3627f2e49f8b4e7cf1521420cd1" member_type: VOTER last_known_addr { host: "127.11.254.189" port: 36155 } attrs { promote: false } } peers { permanent_uuid: "7e4700f28485420fbba29f2e545c71df" member_type: VOTER last_known_addr { host: "127.11.254.188" port: 36857 } attrs { promote: false } } }
I20260812 21:04:00.433254 25416 sys_catalog.cc:458] T 00000000000000000000000000000000 P c90e0f61a5354abbbf96692370a65ebd [sys.catalog]: This master's current role is: LEADER
I20260812 21:04:00.500910 25384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:00.498588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49092 (local address 127.11.254.129:35727)
0812 21:04:00.498655 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0812 21:04:00.498657 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:04:00.498665 (+ 8us) server_negotiation.cc:407] Connection header received
0812 21:04:00.498707 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:04:00.498708 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:04:00.498727 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:04:00.498788 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 21:04:00.498912 (+ 124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:00.499399 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:00.500189 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:00.500402 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:00.500465 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:00.500604 (+ 139us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 21:04:00.500717 (+ 113us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 21:04:00.500789 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0812 21:04:00.500806 (+ 17us) server_negotiation.cc:299] Negotiation successful
0812 21:04:00.500829 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260812 21:04:00.503093 25261 raft_consensus.cc:1261] T 4f9ac0eb8a14441b8294545eb00d1c49 P 228262922b09413f8fe37401b0a932bd [term 1 FOLLOWER]: Refusing update from remote peer 8e16af0efe6c4f698e93d11250de344b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 21:04:00.503093 25036 raft_consensus.cc:1261] T 4f9ac0eb8a14441b8294545eb00d1c49 P a9da4e76358f4ee18755f38b4d8de7ad [term 1 FOLLOWER]: Refusing update from remote peer 8e16af0efe6c4f698e93d11250de344b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 21:04:00.503540 25414 consensus_queue.cc:1035] T 4f9ac0eb8a14441b8294545eb00d1c49 P 8e16af0efe6c4f698e93d11250de344b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9da4e76358f4ee18755f38b4d8de7ad" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 39443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 21:04:00.503685 25408 consensus_queue.cc:1035] T 4f9ac0eb8a14441b8294545eb00d1c49 P 8e16af0efe6c4f698e93d11250de344b [LEADER]: Connected to new peer: Peer: permanent_uuid: "228262922b09413f8fe37401b0a932bd" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 46307 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 21:04:00.508802 25423 mvcc.cc:205] Tried to move back new op lower bound from 7317785151497715712 to 7317785151191252992. Current Snapshot: MvccSnapshot[applied={T|T < 7317785151497715712 or (T in {7317785151497715712})}]
I20260812 21:04:00.509236 23224 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38766:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 21:04:00.509342 23224 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38766:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 21:04:00.510892 23224 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c90e0f61a5354abbbf96692370a65ebd: Sending DeleteTablet for 3 replicas of tablet 4f9ac0eb8a14441b8294545eb00d1c49
I20260812 21:04:00.511286 25235 tablet_service.cc:1514] Processing DeleteTablet for tablet 4f9ac0eb8a14441b8294545eb00d1c49 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:04:00 UTC) from {username='oryx', principal='oryx/127.11.254.190@KRBTEST.COM'} at 127.0.0.1:41230
I20260812 21:04:00.511310 24760 tablet_service.cc:1514] Processing DeleteTablet for tablet 4f9ac0eb8a14441b8294545eb00d1c49 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:04:00 UTC) from {username='oryx', principal='oryx/127.11.254.190@KRBTEST.COM'} at 127.0.0.1:49082
I20260812 21:04:00.511695 25441 tablet_replica.cc:330] T 4f9ac0eb8a14441b8294545eb00d1c49 P 8e16af0efe6c4f698e93d11250de344b: stopping tablet replica
I20260812 21:04:00.511785 25000 tablet_service.cc:1514] Processing DeleteTablet for tablet 4f9ac0eb8a14441b8294545eb00d1c49 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:04:00 UTC) from {username='oryx', principal='oryx/127.11.254.190@KRBTEST.COM'} at 127.0.0.1:57258
I20260812 21:04:00.511936 25441 raft_consensus.cc:2229] T 4f9ac0eb8a14441b8294545eb00d1c49 P 8e16af0efe6c4f698e93d11250de344b [term 1 LEADER]: Raft consensus shutting down.
I20260812 21:04:00.512176 25441 raft_consensus.cc:2258] T 4f9ac0eb8a14441b8294545eb00d1c49 P 8e16af0efe6c4f698e93d11250de344b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 21:04:00.512530 25440 tablet_replica.cc:330] T 4f9ac0eb8a14441b8294545eb00d1c49 P 228262922b09413f8fe37401b0a932bd: stopping tablet replica
I20260812 21:04:00.512630 25440 raft_consensus.cc:2229] T 4f9ac0eb8a14441b8294545eb00d1c49 P 228262922b09413f8fe37401b0a932bd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 21:04:00.512737 25440 raft_consensus.cc:2258] T 4f9ac0eb8a14441b8294545eb00d1c49 P 228262922b09413f8fe37401b0a932bd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 21:04:00.512789 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 23167
I20260812 21:04:00.512809 25441 ts_tablet_manager.cc:1905] T 4f9ac0eb8a14441b8294545eb00d1c49 P 8e16af0efe6c4f698e93d11250de344b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 21:04:00.522078 25440 ts_tablet_manager.cc:1905] T 4f9ac0eb8a14441b8294545eb00d1c49 P 228262922b09413f8fe37401b0a932bd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 21:04:00.524439 25365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:00.512190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32994 (local address 127.11.254.188:36857)
0812 21:04:00.512231 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0812 21:04:00.512233 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:04:00.512508 (+ 275us) server_negotiation.cc:407] Connection header received
0812 21:04:00.512668 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:04:00.512670 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:04:00.512681 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:04:00.512709 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:04:00.517921 (+ 5212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:00.518573 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:00.519402 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:00.519628 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:00.519663 (+ 35us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:00.521277 (+ 1614us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:04:00.521278 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:04:00.521279 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:04:00.521287 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:04:00.523055 (+ 1768us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:04:00.523520 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:04:00.523521 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:04:00.523522 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:04:00.523585 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:04:00.523804 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:04:00.523805 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:04:00.523806 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:04:00.523949 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:04:00.524027 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0812 21:04:00.524255 (+ 228us) server_negotiation.cc:299] Negotiation successful
0812 21:04:00.524346 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 21:04:00.524541 25440 ts_tablet_manager.cc:1918] T 4f9ac0eb8a14441b8294545eb00d1c49 P 228262922b09413f8fe37401b0a932bd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 21:04:00.524619 25440 log.cc:1196] T 4f9ac0eb8a14441b8294545eb00d1c49 P 228262922b09413f8fe37401b0a932bd: Deleting WAL directory at /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/ts-2/wal/wals/4f9ac0eb8a14441b8294545eb00d1c49
I20260812 21:04:00.524891 25440 ts_tablet_manager.cc:1939] T 4f9ac0eb8a14441b8294545eb00d1c49 P 228262922b09413f8fe37401b0a932bd: Deleting consensus metadata
I20260812 21:04:00.526190 25442 tablet_replica.cc:330] T 4f9ac0eb8a14441b8294545eb00d1c49 P a9da4e76358f4ee18755f38b4d8de7ad: stopping tablet replica
I20260812 21:04:00.526408 25442 raft_consensus.cc:2229] T 4f9ac0eb8a14441b8294545eb00d1c49 P a9da4e76358f4ee18755f38b4d8de7ad [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 21:04:00.526603 25442 raft_consensus.cc:2258] T 4f9ac0eb8a14441b8294545eb00d1c49 P a9da4e76358f4ee18755f38b4d8de7ad [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 21:04:00.527323 25442 ts_tablet_manager.cc:1905] T 4f9ac0eb8a14441b8294545eb00d1c49 P a9da4e76358f4ee18755f38b4d8de7ad: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260812 21:04:00.528410 25147 connection.cc:535] server connection from 127.0.0.1:41230 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260812 21:04:00.529305 25442 ts_tablet_manager.cc:1918] T 4f9ac0eb8a14441b8294545eb00d1c49 P a9da4e76358f4ee18755f38b4d8de7ad: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 21:04:00.529424 25442 log.cc:1196] T 4f9ac0eb8a14441b8294545eb00d1c49 P a9da4e76358f4ee18755f38b4d8de7ad: Deleting WAL directory at /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/ts-1/wal/wals/4f9ac0eb8a14441b8294545eb00d1c49
I20260812 21:04:00.529731 25442 ts_tablet_manager.cc:1939] T 4f9ac0eb8a14441b8294545eb00d1c49 P a9da4e76358f4ee18755f38b4d8de7ad: Deleting consensus metadata
W20260812 21:04:00.530185 24907 connection.cc:441] server connection from 127.0.0.1:57258 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57258 (request call id 1) could send its response
I20260812 21:04:00.534075 25441 ts_tablet_manager.cc:1918] T 4f9ac0eb8a14441b8294545eb00d1c49 P 8e16af0efe6c4f698e93d11250de344b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 21:04:00.534154 25441 log.cc:1196] T 4f9ac0eb8a14441b8294545eb00d1c49 P 8e16af0efe6c4f698e93d11250de344b: Deleting WAL directory at /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/ts-0/wal/wals/4f9ac0eb8a14441b8294545eb00d1c49
I20260812 21:04:00.534395 25441 ts_tablet_manager.cc:1939] T 4f9ac0eb8a14441b8294545eb00d1c49 P 8e16af0efe6c4f698e93d11250de344b: Deleting consensus metadata
W20260812 21:04:00.534737 24692 connection.cc:441] server connection from 127.0.0.1:49082 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49082 (request call id 1) could send its response
I20260812 21:04:00.537642 24345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:00.529988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41338 (local address 127.11.254.189:36155)
0812 21:04:00.530031 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0812 21:04:00.530033 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:04:00.530147 (+ 114us) server_negotiation.cc:407] Connection header received
0812 21:04:00.530330 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:04:00.530331 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:04:00.530342 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:04:00.530388 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:04:00.530933 (+ 545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:00.531424 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:00.532319 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:00.532528 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:00.532581 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:00.534549 (+ 1968us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:04:00.534550 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:04:00.534551 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:04:00.534558 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:04:00.536287 (+ 1729us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:04:00.536655 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:04:00.536656 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:04:00.536656 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:04:00.536719 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:04:00.536893 (+ 174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:04:00.536893 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:04:00.536894 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:04:00.537026 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:04:00.537116 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0812 21:04:00.537346 (+ 230us) server_negotiation.cc:299] Negotiation successful
0812 21:04:00.537443 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260812 21:04:00.541148 25365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:00.534908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32998 (local address 127.11.254.188:36857)
0812 21:04:00.534948 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0812 21:04:00.534950 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:04:00.535055 (+ 105us) server_negotiation.cc:407] Connection header received
0812 21:04:00.535123 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:04:00.535124 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:04:00.535133 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:04:00.535177 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:04:00.535706 (+ 529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:00.536182 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:00.537066 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:00.537267 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:00.537321 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:00.538583 (+ 1262us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:04:00.538584 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:04:00.538585 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:04:00.538592 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:04:00.539956 (+ 1364us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:04:00.540254 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:04:00.540255 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:04:00.540256 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:04:00.540314 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:04:00.540550 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:04:00.540551 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:04:00.540551 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:04:00.540757 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:04:00.540871 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0812 21:04:00.540960 (+ 89us) server_negotiation.cc:299] Negotiation successful
0812 21:04:00.541048 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260812 21:04:01.507443 25455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:01.507065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:43007 (local address 127.11.254.130:47027)
0812 21:04:01.507243 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0812 21:04:01.507379 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.190:43007: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":91,"threads_started":1}
W20260812 21:04:01.507717 25114 heartbeater.cc:644] Failed to heartbeat to 127.11.254.190:43007 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.11.254.190:43007: connect: Connection refused (error 111)
I20260812 21:04:01.509627 25456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:01.509343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:43007 (local address 127.11.254.131:55757)
0812 21:04:01.509499 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0812 21:04:01.509578 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.190:43007: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":80,"threads_started":1}
I20260812 21:04:01.509809 25457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:01.509492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:43007 (local address 127.11.254.129:55117)
0812 21:04:01.509632 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0812 21:04:01.509763 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.190:43007: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":46,"threads_started":1}
W20260812 21:04:01.509821 25339 heartbeater.cc:644] Failed to heartbeat to 127.11.254.190:43007 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.11.254.190:43007: connect: Connection refused (error 111)
W20260812 21:04:01.510010 24867 heartbeater.cc:644] Failed to heartbeat to 127.11.254.190:43007 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.11.254.190:43007: connect: Connection refused (error 111)
I20260812 21:04:02.011642 25458 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5026b3627f2e49f8b4e7cf1521420cd1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c90e0f61a5354abbbf96692370a65ebd)
I20260812 21:04:02.011683 25462 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7e4700f28485420fbba29f2e545c71df [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c90e0f61a5354abbbf96692370a65ebd)
I20260812 21:04:02.011803 25458 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5026b3627f2e49f8b4e7cf1521420cd1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "c90e0f61a5354abbbf96692370a65ebd" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 43007 } } peers { permanent_uuid: "5026b3627f2e49f8b4e7cf1521420cd1" member_type: VOTER last_known_addr { host: "127.11.254.189" port: 36155 } attrs { promote: false } } peers { permanent_uuid: "7e4700f28485420fbba29f2e545c71df" member_type: VOTER last_known_addr { host: "127.11.254.188" port: 36857 } attrs { promote: false } }
I20260812 21:04:02.011802 25462 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7e4700f28485420fbba29f2e545c71df [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "c90e0f61a5354abbbf96692370a65ebd" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 43007 } } peers { permanent_uuid: "5026b3627f2e49f8b4e7cf1521420cd1" member_type: VOTER last_known_addr { host: "127.11.254.189" port: 36155 } attrs { promote: false } } peers { permanent_uuid: "7e4700f28485420fbba29f2e545c71df" member_type: VOTER last_known_addr { host: "127.11.254.188" port: 36857 } attrs { promote: false } }
I20260812 21:04:02.012066 25462 leader_election.cc:288] T 00000000000000000000000000000000 P 7e4700f28485420fbba29f2e545c71df [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c90e0f61a5354abbbf96692370a65ebd (127.11.254.190:43007), 5026b3627f2e49f8b4e7cf1521420cd1 (127.11.254.189:36155)
I20260812 21:04:02.012308 25458 leader_election.cc:288] T 00000000000000000000000000000000 P 5026b3627f2e49f8b4e7cf1521420cd1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c90e0f61a5354abbbf96692370a65ebd (127.11.254.190:43007), 7e4700f28485420fbba29f2e545c71df (127.11.254.188:36857)
I20260812 21:04:02.012576 25466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:02.012273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:43007 (local address 127.0.0.1:39256)
0812 21:04:02.012477 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0812 21:04:02.012541 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.190:43007: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":56,"threads_started":1}
I20260812 21:04:02.012580 25463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:02.012216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:43007 (local address 127.0.0.1:39244)
0812 21:04:02.012405 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0812 21:04:02.012533 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.190:43007: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":58,"threads_started":1}
W20260812 21:04:02.012708 24558 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.254.190:43007: connect: Connection refused (error 111)
W20260812 21:04:02.012708 24218 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.254.190:43007: connect: Connection refused (error 111)
I20260812 21:04:02.013139 25463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:02.013030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:43007 (local address 127.0.0.1:39270)
0812 21:04:02.013080 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0812 21:04:02.013112 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.190:43007: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260812 21:04:02.013183 25466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:02.013030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:43007 (local address 127.0.0.1:39274)
0812 21:04:02.013104 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0812 21:04:02.013157 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.190:43007: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260812 21:04:02.013311 24218 leader_election.cc:334] T 00000000000000000000000000000000 P 5026b3627f2e49f8b4e7cf1521420cd1 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c90e0f61a5354abbbf96692370a65ebd (127.11.254.190:43007): Network error: Client connection negotiation failed: client connection to 127.11.254.190:43007: connect: Connection refused (error 111)
W20260812 21:04:02.013331 24558 leader_election.cc:334] T 00000000000000000000000000000000 P 7e4700f28485420fbba29f2e545c71df [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c90e0f61a5354abbbf96692370a65ebd (127.11.254.190:43007): Network error: Client connection negotiation failed: client connection to 127.11.254.190:43007: connect: Connection refused (error 111)
I20260812 21:04:02.015560 25469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:02.012553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33000 (local address 127.11.254.188:36857)
0812 21:04:02.012633 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0812 21:04:02.012635 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:04:02.012643 (+ 8us) server_negotiation.cc:407] Connection header received
0812 21:04:02.012672 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:04:02.012673 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:04:02.012683 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:04:02.012928 (+ 245us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:04:02.012966 (+ 38us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:02.013630 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:02.014785 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:02.015289 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:02.015340 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:02.015348 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 21:04:02.015480 (+ 132us) server_negotiation.cc:299] Negotiation successful
0812 21:04:02.015505 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69,"thread_start_us":45,"threads_started":1}
I20260812 21:04:02.015588 25465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:02.012231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.188:36857 (local address 127.0.0.1:33000)
0812 21:04:02.012321 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0812 21:04:02.012334 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 21:04:02.012396 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:04:02.012787 (+ 391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:04:02.012789 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:04:02.012792 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:04:02.012864 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:02.012866 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:02.013710 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:02.013711 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:02.014707 (+ 996us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:02.014709 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:02.015351 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:02.015352 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:02.015415 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:02.015419 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 21:04:02.015475 (+ 56us) client_negotiation.cc:240] Negotiation successful
0812 21:04:02.015518 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":68,"thread_start_us":41,"threads_started":1}
I20260812 21:04:02.015810 24597 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5026b3627f2e49f8b4e7cf1521420cd1" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "7e4700f28485420fbba29f2e545c71df" is_pre_election: true
I20260812 21:04:02.015933 24597 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 7e4700f28485420fbba29f2e545c71df [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5026b3627f2e49f8b4e7cf1521420cd1 in term 1.
I20260812 21:04:02.015991 25468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:02.012444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41350 (local address 127.11.254.189:36155)
0812 21:04:02.012937 (+ 493us) server_negotiation.cc:206] Beginning negotiation
0812 21:04:02.012939 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:04:02.012949 (+ 10us) server_negotiation.cc:407] Connection header received
0812 21:04:02.013067 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:04:02.013068 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:04:02.013079 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:04:02.013411 (+ 332us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:04:02.013448 (+ 37us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:02.014091 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:02.015191 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:02.015661 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:02.015753 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:02.015762 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 21:04:02.015894 (+ 132us) server_negotiation.cc:299] Negotiation successful
0812 21:04:02.015934 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":478,"thread_start_us":35,"threads_started":1}
I20260812 21:04:02.016003 25467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:02.012367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.189:36155 (local address 127.0.0.1:41350)
0812 21:04:02.012599 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0812 21:04:02.012610 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 21:04:02.012819 (+ 209us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:04:02.013179 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:04:02.013180 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:04:02.013183 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:04:02.013239 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:02.013241 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:02.014162 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:02.014163 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:02.015111 (+ 948us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:02.015113 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:02.015758 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:02.015760 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:02.015819 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:02.015824 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 21:04:02.015876 (+ 52us) client_negotiation.cc:240] Negotiation successful
0812 21:04:02.015919 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":53,"threads_started":1}
I20260812 21:04:02.016129 24220 leader_election.cc:302] T 00000000000000000000000000000000 P 5026b3627f2e49f8b4e7cf1521420cd1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5026b3627f2e49f8b4e7cf1521420cd1, 7e4700f28485420fbba29f2e545c71df; no voters: c90e0f61a5354abbbf96692370a65ebd
I20260812 21:04:02.016179 24262 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7e4700f28485420fbba29f2e545c71df" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "5026b3627f2e49f8b4e7cf1521420cd1" is_pre_election: true
I20260812 21:04:02.016233 25458 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5026b3627f2e49f8b4e7cf1521420cd1 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260812 21:04:02.016278 24262 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 5026b3627f2e49f8b4e7cf1521420cd1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e4700f28485420fbba29f2e545c71df in term 1.
I20260812 21:04:02.016326 25458 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5026b3627f2e49f8b4e7cf1521420cd1 [term 1 FOLLOWER]: Starting leader election (detected failure of leader c90e0f61a5354abbbf96692370a65ebd)
I20260812 21:04:02.016355 25458 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5026b3627f2e49f8b4e7cf1521420cd1 [term 1 FOLLOWER]: Advancing to term 2
I20260812 21:04:02.016464 24558 leader_election.cc:302] T 00000000000000000000000000000000 P 7e4700f28485420fbba29f2e545c71df [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5026b3627f2e49f8b4e7cf1521420cd1, 7e4700f28485420fbba29f2e545c71df; no voters: c90e0f61a5354abbbf96692370a65ebd
I20260812 21:04:02.016578 25462 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7e4700f28485420fbba29f2e545c71df [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260812 21:04:02.016625 25462 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7e4700f28485420fbba29f2e545c71df [term 1 FOLLOWER]: Starting leader election (detected failure of leader c90e0f61a5354abbbf96692370a65ebd)
I20260812 21:04:02.016654 25462 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7e4700f28485420fbba29f2e545c71df [term 1 FOLLOWER]: Advancing to term 2
I20260812 21:04:02.017131 25458 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5026b3627f2e49f8b4e7cf1521420cd1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "c90e0f61a5354abbbf96692370a65ebd" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 43007 } } peers { permanent_uuid: "5026b3627f2e49f8b4e7cf1521420cd1" member_type: VOTER last_known_addr { host: "127.11.254.189" port: 36155 } attrs { promote: false } } peers { permanent_uuid: "7e4700f28485420fbba29f2e545c71df" member_type: VOTER last_known_addr { host: "127.11.254.188" port: 36857 } attrs { promote: false } }
I20260812 21:04:02.017259 25458 leader_election.cc:288] T 00000000000000000000000000000000 P 5026b3627f2e49f8b4e7cf1521420cd1 [CANDIDATE]: Term 2 election: Requested vote from peers c90e0f61a5354abbbf96692370a65ebd (127.11.254.190:43007), 7e4700f28485420fbba29f2e545c71df (127.11.254.188:36857)
I20260812 21:04:02.017342 25462 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7e4700f28485420fbba29f2e545c71df [term 2 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "c90e0f61a5354abbbf96692370a65ebd" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 43007 } } peers { permanent_uuid: "5026b3627f2e49f8b4e7cf1521420cd1" member_type: VOTER last_known_addr { host: "127.11.254.189" port: 36155 } attrs { promote: false } } peers { permanent_uuid: "7e4700f28485420fbba29f2e545c71df" member_type: VOTER last_known_addr { host: "127.11.254.188" port: 36857 } attrs { promote: false } }
I20260812 21:04:02.017396 24597 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5026b3627f2e49f8b4e7cf1521420cd1" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "7e4700f28485420fbba29f2e545c71df"
I20260812 21:04:02.017467 25462 leader_election.cc:288] T 00000000000000000000000000000000 P 7e4700f28485420fbba29f2e545c71df [CANDIDATE]: Term 2 election: Requested vote from peers c90e0f61a5354abbbf96692370a65ebd (127.11.254.190:43007), 5026b3627f2e49f8b4e7cf1521420cd1 (127.11.254.189:36155)
I20260812 21:04:02.017473 25465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:02.017379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:43007 (local address 127.0.0.1:39282)
0812 21:04:02.017418 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0812 21:04:02.017450 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.190:43007: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
I20260812 21:04:02.017482 24597 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 7e4700f28485420fbba29f2e545c71df [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 5026b3627f2e49f8b4e7cf1521420cd1 in current term 2: Already voted for candidate 7e4700f28485420fbba29f2e545c71df in this term.
I20260812 21:04:02.017812 25467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:02.017732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:43007 (local address 127.0.0.1:39300)
0812 21:04:02.017768 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0812 21:04:02.017792 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.190:43007: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260812 21:04:02.018091 24262 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7e4700f28485420fbba29f2e545c71df" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "5026b3627f2e49f8b4e7cf1521420cd1"
I20260812 21:04:02.018162 25467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:02.018084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:43007 (local address 127.0.0.1:39308)
0812 21:04:02.018117 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0812 21:04:02.018141 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.190:43007: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
I20260812 21:04:02.018183 24262 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 5026b3627f2e49f8b4e7cf1521420cd1 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e4700f28485420fbba29f2e545c71df in current term 2: Already voted for candidate 5026b3627f2e49f8b4e7cf1521420cd1 in this term.
W20260812 21:04:02.018231 24558 leader_election.cc:334] T 00000000000000000000000000000000 P 7e4700f28485420fbba29f2e545c71df [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer c90e0f61a5354abbbf96692370a65ebd (127.11.254.190:43007): Network error: Client connection negotiation failed: client connection to 127.11.254.190:43007: connect: Connection refused (error 111)
I20260812 21:04:02.018272 25465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:02.018197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:43007 (local address 127.0.0.1:39322)
0812 21:04:02.018232 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0812 21:04:02.018254 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.190:43007: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
I20260812 21:04:02.018322 24558 leader_election.cc:302] T 00000000000000000000000000000000 P 7e4700f28485420fbba29f2e545c71df [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e4700f28485420fbba29f2e545c71df; no voters: 5026b3627f2e49f8b4e7cf1521420cd1, c90e0f61a5354abbbf96692370a65ebd
W20260812 21:04:02.018321 24218 leader_election.cc:334] T 00000000000000000000000000000000 P 5026b3627f2e49f8b4e7cf1521420cd1 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer c90e0f61a5354abbbf96692370a65ebd (127.11.254.190:43007): Network error: Client connection negotiation failed: client connection to 127.11.254.190:43007: connect: Connection refused (error 111)
I20260812 21:04:02.018356 24218 leader_election.cc:302] T 00000000000000000000000000000000 P 5026b3627f2e49f8b4e7cf1521420cd1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5026b3627f2e49f8b4e7cf1521420cd1; no voters: 7e4700f28485420fbba29f2e545c71df, c90e0f61a5354abbbf96692370a65ebd
I20260812 21:04:02.018402 25462 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 7e4700f28485420fbba29f2e545c71df [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260812 21:04:02.018476 25458 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 5026b3627f2e49f8b4e7cf1521420cd1 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260812 21:04:02.508435 25475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:02.508144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:43007 (local address 127.11.254.130:44533)
0812 21:04:02.508309 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0812 21:04:02.508363 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.190:43007: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":99,"threads_started":1}
I20260812 21:04:02.510478 25478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:02.510288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:43007 (local address 127.11.254.129:42015)
0812 21:04:02.510404 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0812 21:04:02.510438 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.190:43007: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":64,"threads_started":1}
I20260812 21:04:02.510556 25476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:02.510117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:43007 (local address 127.11.254.131:48885)
0812 21:04:02.510447 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0812 21:04:02.510487 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.190:43007: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":310,"thread_start_us":75,"threads_started":1}
I20260812 21:04:03.509303 25479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:03.509053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:43007 (local address 127.11.254.130:43743)
0812 21:04:03.509199 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0812 21:04:03.509243 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.190:43007: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":73,"threads_started":1}
I20260812 21:04:03.511166 25480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:03.510914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:43007 (local address 127.11.254.129:39195)
0812 21:04:03.511091 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0812 21:04:03.511123 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.190:43007: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":63,"threads_started":1}
I20260812 21:04:03.511353 25481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:03.510919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:43007 (local address 127.11.254.131:39243)
0812 21:04:03.511277 (+ 358us) negotiation.cc:107] Waiting for socket to connect
0812 21:04:03.511312 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.190:43007: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":341,"thread_start_us":62,"threads_started":1}
I20260812 21:04:03.569569 25482 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7e4700f28485420fbba29f2e545c71df [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 21:04:03.569703 25482 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7e4700f28485420fbba29f2e545c71df [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "c90e0f61a5354abbbf96692370a65ebd" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 43007 } } peers { permanent_uuid: "5026b3627f2e49f8b4e7cf1521420cd1" member_type: VOTER last_known_addr { host: "127.11.254.189" port: 36155 } attrs { promote: false } } peers { permanent_uuid: "7e4700f28485420fbba29f2e545c71df" member_type: VOTER last_known_addr { host: "127.11.254.188" port: 36857 } attrs { promote: false } }
I20260812 21:04:03.569929 25482 leader_election.cc:288] T 00000000000000000000000000000000 P 7e4700f28485420fbba29f2e545c71df [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers c90e0f61a5354abbbf96692370a65ebd (127.11.254.190:43007), 5026b3627f2e49f8b4e7cf1521420cd1 (127.11.254.189:36155)
I20260812 21:04:03.570281 25489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:03.570093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:43007 (local address 127.0.0.1:39544)
0812 21:04:03.570192 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0812 21:04:03.570236 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.190:43007: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79,"thread_start_us":47,"threads_started":1}
I20260812 21:04:03.570400 24262 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7e4700f28485420fbba29f2e545c71df" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "5026b3627f2e49f8b4e7cf1521420cd1" is_pre_election: true
I20260812 21:04:03.570533 24262 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 5026b3627f2e49f8b4e7cf1521420cd1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e4700f28485420fbba29f2e545c71df in term 2.
I20260812 21:04:03.570729 24558 leader_election.cc:302] T 00000000000000000000000000000000 P 7e4700f28485420fbba29f2e545c71df [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5026b3627f2e49f8b4e7cf1521420cd1, 7e4700f28485420fbba29f2e545c71df; no voters:
I20260812 21:04:03.570776 25489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:03.570673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:43007 (local address 127.0.0.1:39554)
0812 21:04:03.570718 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0812 21:04:03.570747 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.190:43007: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260812 21:04:03.570818 24558 leader_election.cc:334] T 00000000000000000000000000000000 P 7e4700f28485420fbba29f2e545c71df [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer c90e0f61a5354abbbf96692370a65ebd (127.11.254.190:43007): Network error: Client connection negotiation failed: client connection to 127.11.254.190:43007: connect: Connection refused (error 111)
I20260812 21:04:03.570852 25482 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7e4700f28485420fbba29f2e545c71df [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260812 21:04:03.570904 25482 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7e4700f28485420fbba29f2e545c71df [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 21:04:03.570938 25482 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7e4700f28485420fbba29f2e545c71df [term 2 FOLLOWER]: Advancing to term 3
I20260812 21:04:03.571720 25482 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7e4700f28485420fbba29f2e545c71df [term 3 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "c90e0f61a5354abbbf96692370a65ebd" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 43007 } } peers { permanent_uuid: "5026b3627f2e49f8b4e7cf1521420cd1" member_type: VOTER last_known_addr { host: "127.11.254.189" port: 36155 } attrs { promote: false } } peers { permanent_uuid: "7e4700f28485420fbba29f2e545c71df" member_type: VOTER last_known_addr { host: "127.11.254.188" port: 36857 } attrs { promote: false } }
I20260812 21:04:03.571856 25482 leader_election.cc:288] T 00000000000000000000000000000000 P 7e4700f28485420fbba29f2e545c71df [CANDIDATE]: Term 3 election: Requested vote from peers c90e0f61a5354abbbf96692370a65ebd (127.11.254.190:43007), 5026b3627f2e49f8b4e7cf1521420cd1 (127.11.254.189:36155)
I20260812 21:04:03.572125 25489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:03.571993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:43007 (local address 127.0.0.1:39558)
0812 21:04:03.572041 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0812 21:04:03.572072 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.190:43007: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260812 21:04:03.572232 24262 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7e4700f28485420fbba29f2e545c71df" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "5026b3627f2e49f8b4e7cf1521420cd1"
I20260812 21:04:03.572350 24262 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5026b3627f2e49f8b4e7cf1521420cd1 [term 2 FOLLOWER]: Advancing to term 3
I20260812 21:04:03.572480 25489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:03.572402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:43007 (local address 127.0.0.1:39560)
0812 21:04:03.572431 (+ 29us) negotiation.cc:107] Waiting for socket to connect
0812 21:04:03.572458 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.190:43007: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":14}
W20260812 21:04:03.572551 24558 leader_election.cc:334] T 00000000000000000000000000000000 P 7e4700f28485420fbba29f2e545c71df [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer c90e0f61a5354abbbf96692370a65ebd (127.11.254.190:43007): Network error: Client connection negotiation failed: client connection to 127.11.254.190:43007: connect: Connection refused (error 111)
I20260812 21:04:03.573072 24262 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 5026b3627f2e49f8b4e7cf1521420cd1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e4700f28485420fbba29f2e545c71df in term 3.
I20260812 21:04:03.573264 24558 leader_election.cc:302] T 00000000000000000000000000000000 P 7e4700f28485420fbba29f2e545c71df [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5026b3627f2e49f8b4e7cf1521420cd1, 7e4700f28485420fbba29f2e545c71df; no voters: c90e0f61a5354abbbf96692370a65ebd
I20260812 21:04:03.573374 25482 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7e4700f28485420fbba29f2e545c71df [term 3 FOLLOWER]: Leader election won for term 3
I20260812 21:04:03.573487 25482 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7e4700f28485420fbba29f2e545c71df [term 3 LEADER]: Becoming Leader. State: Replica: 7e4700f28485420fbba29f2e545c71df, State: Running, Role: LEADER
I20260812 21:04:03.573552 25482 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7e4700f28485420fbba29f2e545c71df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "c90e0f61a5354abbbf96692370a65ebd" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 43007 } } peers { permanent_uuid: "5026b3627f2e49f8b4e7cf1521420cd1" member_type: VOTER last_known_addr { host: "127.11.254.189" port: 36155 } attrs { promote: false } } peers { permanent_uuid: "7e4700f28485420fbba29f2e545c71df" member_type: VOTER last_known_addr { host: "127.11.254.188" port: 36857 } attrs { promote: false } }
I20260812 21:04:03.573925 25493 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e4700f28485420fbba29f2e545c71df [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7e4700f28485420fbba29f2e545c71df. Latest consensus state: current_term: 3 leader_uuid: "7e4700f28485420fbba29f2e545c71df" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "c90e0f61a5354abbbf96692370a65ebd" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 43007 } } peers { permanent_uuid: "5026b3627f2e49f8b4e7cf1521420cd1" member_type: VOTER last_known_addr { host: "127.11.254.189" port: 36155 } attrs { promote: false } } peers { permanent_uuid: "7e4700f28485420fbba29f2e545c71df" member_type: VOTER last_known_addr { host: "127.11.254.188" port: 36857 } attrs { promote: false } } }
I20260812 21:04:03.574048 25493 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e4700f28485420fbba29f2e545c71df [sys.catalog]: This master's current role is: LEADER
I20260812 21:04:03.574234 25496 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 21:04:03.574689 25496 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 21:04:03.574771 25496 catalog_manager.cc:1259] Loaded cluster ID: b39e89298b2f46b5a2e07cc9a4507645
I20260812 21:04:03.574802 25496 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 21:04:03.574970 25496 catalog_manager.cc:1504] Loading token signing keys...
I20260812 21:04:03.575037 25496 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 7e4700f28485420fbba29f2e545c71df: Loaded TSK: 0
I20260812 21:04:03.575172 25496 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 21:04:03.628264 24582 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32998:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 21:04:03.630132 24582 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 21:04:03.631202 25489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:03.631086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:43007 (local address 127.0.0.1:39574)
0812 21:04:03.631133 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0812 21:04:03.631168 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.190:43007: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260812 21:04:03.631335 24262 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5026b3627f2e49f8b4e7cf1521420cd1 [term 3 FOLLOWER]: Refusing update from remote peer 7e4700f28485420fbba29f2e545c71df: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260812 21:04:03.631645 25489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:03.631555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:43007 (local address 127.0.0.1:39576)
0812 21:04:03.631593 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0812 21:04:03.631622 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.190:43007: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260812 21:04:03.631644 25482 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 7e4700f28485420fbba29f2e545c71df [LEADER]: Connected to new peer: Peer: permanent_uuid: "5026b3627f2e49f8b4e7cf1521420cd1" member_type: VOTER last_known_addr { host: "127.11.254.189" port: 36155 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260812 21:04:03.631778 24558 consensus_peers.cc:487] T 00000000000000000000000000000000 P 7e4700f28485420fbba29f2e545c71df -> Peer c90e0f61a5354abbbf96692370a65ebd (127.11.254.190:43007): Couldn't send request to peer c90e0f61a5354abbbf96692370a65ebd. Status: Network error: Client connection negotiation failed: client connection to 127.11.254.190:43007: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260812 21:04:03.632265 25498 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5026b3627f2e49f8b4e7cf1521420cd1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7e4700f28485420fbba29f2e545c71df. Latest consensus state: current_term: 3 leader_uuid: "7e4700f28485420fbba29f2e545c71df" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "c90e0f61a5354abbbf96692370a65ebd" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 43007 } } peers { permanent_uuid: "5026b3627f2e49f8b4e7cf1521420cd1" member_type: VOTER last_known_addr { host: "127.11.254.189" port: 36155 } attrs { promote: false } } peers { permanent_uuid: "7e4700f28485420fbba29f2e545c71df" member_type: VOTER last_known_addr { host: "127.11.254.188" port: 36857 } attrs { promote: false } } }
I20260812 21:04:03.632361 25498 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5026b3627f2e49f8b4e7cf1521420cd1 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 21:04:03.632817 25482 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e4700f28485420fbba29f2e545c71df [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "7e4700f28485420fbba29f2e545c71df" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "c90e0f61a5354abbbf96692370a65ebd" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 43007 } } peers { permanent_uuid: "5026b3627f2e49f8b4e7cf1521420cd1" member_type: VOTER last_known_addr { host: "127.11.254.189" port: 36155 } attrs { promote: false } } peers { permanent_uuid: "7e4700f28485420fbba29f2e545c71df" member_type: VOTER last_known_addr { host: "127.11.254.188" port: 36857 } attrs { promote: false } } }
I20260812 21:04:03.632901 25482 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e4700f28485420fbba29f2e545c71df [sys.catalog]: This master's current role is: LEADER
I20260812 21:04:03.632982 25498 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5026b3627f2e49f8b4e7cf1521420cd1 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "7e4700f28485420fbba29f2e545c71df" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "c90e0f61a5354abbbf96692370a65ebd" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 43007 } } peers { permanent_uuid: "5026b3627f2e49f8b4e7cf1521420cd1" member_type: VOTER last_known_addr { host: "127.11.254.189" port: 36155 } attrs { promote: false } } peers { permanent_uuid: "7e4700f28485420fbba29f2e545c71df" member_type: VOTER last_known_addr { host: "127.11.254.188" port: 36857 } attrs { promote: false } } }
I20260812 21:04:03.633075 25498 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5026b3627f2e49f8b4e7cf1521420cd1 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 21:04:03.638118 25509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:03.634882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.129:35727 (local address 127.0.0.1:49102)
0812 21:04:03.635052 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0812 21:04:03.635061 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 21:04:03.635099 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:04:03.635426 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:04:03.635426 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:04:03.635429 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:04:03.635488 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:03.635490 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:03.636126 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:03.636127 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:03.637206 (+ 1079us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:03.637209 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:03.637850 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:03.637851 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:03.637912 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:03.637916 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 21:04:03.637984 (+ 68us) client_negotiation.cc:240] Negotiation successful
0812 21:04:03.638038 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"mutex_wait_us":1,"thread_start_us":100,"threads_started":1}
I20260812 21:04:03.638118 25516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:03.635112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49102 (local address 127.11.254.129:35727)
0812 21:04:03.635257 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0812 21:04:03.635259 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:04:03.635268 (+ 9us) server_negotiation.cc:407] Connection header received
0812 21:04:03.635293 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:04:03.635294 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:04:03.635304 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:04:03.635372 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:04:03.635563 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:03.636053 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:03.637304 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:03.637784 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:03.637845 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:03.637855 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0812 21:04:03.637990 (+ 135us) server_negotiation.cc:299] Negotiation successful
0812 21:04:03.638038 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":71,"threads_started":1}
I20260812 21:04:03.638262 25515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:03.635120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57260 (local address 127.11.254.130:39443)
0812 21:04:03.635248 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0812 21:04:03.635250 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:04:03.635258 (+ 8us) server_negotiation.cc:407] Connection header received
0812 21:04:03.635283 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:04:03.635284 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:04:03.635294 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:04:03.635522 (+ 228us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:04:03.635558 (+ 36us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:03.636304 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:03.637332 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:03.637920 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:03.638023 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:03.638033 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0812 21:04:03.638150 (+ 117us) server_negotiation.cc:299] Negotiation successful
0812 21:04:03.638188 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":71,"threads_started":1}
I20260812 21:04:03.638278 25506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:03.634929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.130:39443 (local address 127.0.0.1:57260)
0812 21:04:03.635072 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0812 21:04:03.635082 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 21:04:03.635134 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:04:03.635414 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:04:03.635415 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:04:03.635417 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:04:03.635461 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:03.635463 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:03.636411 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:03.636412 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:03.637203 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:03.637205 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:03.638020 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:03.638021 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:03.638084 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:03.638088 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 21:04:03.638145 (+ 57us) client_negotiation.cc:240] Negotiation successful
0812 21:04:03.638188 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"mutex_wait_us":16,"thread_start_us":83,"threads_started":1}
I20260812 21:04:03.638347 24760 tablet_service.cc:1467] Processing CreateTablet for tablet 46376ec015b9430eaf1ec4f4eeb41cf6 (DEFAULT_TABLE table=test-table [id=e292eabdcd264763bf14981d26550474]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:04:03.638491 24760 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 46376ec015b9430eaf1ec4f4eeb41cf6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:04:03.638538 25000 tablet_service.cc:1467] Processing CreateTablet for tablet 46376ec015b9430eaf1ec4f4eeb41cf6 (DEFAULT_TABLE table=test-table [id=e292eabdcd264763bf14981d26550474]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:04:03.638648 25000 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 46376ec015b9430eaf1ec4f4eeb41cf6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:04:03.639101 25489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:03.634851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.131:46307 (local address 127.0.0.1:41234)
0812 21:04:03.634885 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0812 21:04:03.634896 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 21:04:03.634949 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:04:03.635393 (+ 444us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:04:03.635394 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:04:03.635396 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:04:03.635463 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:03.635463 (+ 0us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:03.636383 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:03.636384 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:03.637461 (+ 1077us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:03.637463 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:03.638865 (+ 1402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:03.638866 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:03.638939 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:03.638943 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 21:04:03.638997 (+ 54us) client_negotiation.cc:240] Negotiation successful
0812 21:04:03.639038 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17,"mutex_wait_us":4}
I20260812 21:04:03.639115 25510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:03.634961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41234 (local address 127.11.254.131:46307)
0812 21:04:03.635248 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0812 21:04:03.635250 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:04:03.635258 (+ 8us) server_negotiation.cc:407] Connection header received
0812 21:04:03.635284 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:04:03.635284 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:04:03.635295 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:04:03.635523 (+ 228us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:04:03.635559 (+ 36us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:03.636313 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:03.637541 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:03.638786 (+ 1245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:03.638857 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:03.638866 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 21:04:03.639006 (+ 140us) server_negotiation.cc:299] Negotiation successful
0812 21:04:03.639049 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":265,"thread_start_us":52,"threads_started":1}
I20260812 21:04:03.639505 25235 tablet_service.cc:1467] Processing CreateTablet for tablet 46376ec015b9430eaf1ec4f4eeb41cf6 (DEFAULT_TABLE table=test-table [id=e292eabdcd264763bf14981d26550474]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:04:03.639655 25235 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 46376ec015b9430eaf1ec4f4eeb41cf6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:04:03.639976 25520 tablet_bootstrap.cc:492] T 46376ec015b9430eaf1ec4f4eeb41cf6 P 8e16af0efe6c4f698e93d11250de344b: Bootstrap starting.
I20260812 21:04:03.640130 25521 tablet_bootstrap.cc:492] T 46376ec015b9430eaf1ec4f4eeb41cf6 P a9da4e76358f4ee18755f38b4d8de7ad: Bootstrap starting.
I20260812 21:04:03.640475 25521 tablet_bootstrap.cc:654] T 46376ec015b9430eaf1ec4f4eeb41cf6 P a9da4e76358f4ee18755f38b4d8de7ad: Neither blocks nor log segments found. Creating new log.
I20260812 21:04:03.641153 25521 tablet_bootstrap.cc:492] T 46376ec015b9430eaf1ec4f4eeb41cf6 P a9da4e76358f4ee18755f38b4d8de7ad: No bootstrap required, opened a new log
I20260812 21:04:03.641181 25525 tablet_bootstrap.cc:492] T 46376ec015b9430eaf1ec4f4eeb41cf6 P 228262922b09413f8fe37401b0a932bd: Bootstrap starting.
I20260812 21:04:03.641222 25520 tablet_bootstrap.cc:654] T 46376ec015b9430eaf1ec4f4eeb41cf6 P 8e16af0efe6c4f698e93d11250de344b: Neither blocks nor log segments found. Creating new log.
I20260812 21:04:03.641237 25521 ts_tablet_manager.cc:1397] T 46376ec015b9430eaf1ec4f4eeb41cf6 P a9da4e76358f4ee18755f38b4d8de7ad: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 21:04:03.641501 25521 raft_consensus.cc:348] T 46376ec015b9430eaf1ec4f4eeb41cf6 P a9da4e76358f4ee18755f38b4d8de7ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "228262922b09413f8fe37401b0a932bd" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 46307 } } peers { permanent_uuid: "8e16af0efe6c4f698e93d11250de344b" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 35727 } } peers { permanent_uuid: "a9da4e76358f4ee18755f38b4d8de7ad" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 39443 } }
I20260812 21:04:03.641623 25521 raft_consensus.cc:374] T 46376ec015b9430eaf1ec4f4eeb41cf6 P a9da4e76358f4ee18755f38b4d8de7ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:04:03.641628 25525 tablet_bootstrap.cc:654] T 46376ec015b9430eaf1ec4f4eeb41cf6 P 228262922b09413f8fe37401b0a932bd: Neither blocks nor log segments found. Creating new log.
I20260812 21:04:03.641659 25521 raft_consensus.cc:729] T 46376ec015b9430eaf1ec4f4eeb41cf6 P a9da4e76358f4ee18755f38b4d8de7ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9da4e76358f4ee18755f38b4d8de7ad, State: Initialized, Role: FOLLOWER
I20260812 21:04:03.641728 25521 consensus_queue.cc:260] T 46376ec015b9430eaf1ec4f4eeb41cf6 P a9da4e76358f4ee18755f38b4d8de7ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "228262922b09413f8fe37401b0a932bd" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 46307 } } peers { permanent_uuid: "8e16af0efe6c4f698e93d11250de344b" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 35727 } } peers { permanent_uuid: "a9da4e76358f4ee18755f38b4d8de7ad" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 39443 } }
I20260812 21:04:03.641925 25521 ts_tablet_manager.cc:1428] T 46376ec015b9430eaf1ec4f4eeb41cf6 P a9da4e76358f4ee18755f38b4d8de7ad: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260812 21:04:03.641981 25520 tablet_bootstrap.cc:492] T 46376ec015b9430eaf1ec4f4eeb41cf6 P 8e16af0efe6c4f698e93d11250de344b: No bootstrap required, opened a new log
I20260812 21:04:03.642028 25520 ts_tablet_manager.cc:1397] T 46376ec015b9430eaf1ec4f4eeb41cf6 P 8e16af0efe6c4f698e93d11250de344b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 21:04:03.642230 25479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:03.642089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:43007 (local address 127.11.254.130:40229)
0812 21:04:03.642169 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0812 21:04:03.642200 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.190:43007: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260812 21:04:03.642227 25520 raft_consensus.cc:348] T 46376ec015b9430eaf1ec4f4eeb41cf6 P 8e16af0efe6c4f698e93d11250de344b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "228262922b09413f8fe37401b0a932bd" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 46307 } } peers { permanent_uuid: "8e16af0efe6c4f698e93d11250de344b" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 35727 } } peers { permanent_uuid: "a9da4e76358f4ee18755f38b4d8de7ad" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 39443 } }
I20260812 21:04:03.642302 25520 raft_consensus.cc:374] T 46376ec015b9430eaf1ec4f4eeb41cf6 P 8e16af0efe6c4f698e93d11250de344b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:04:03.642321 25520 raft_consensus.cc:729] T 46376ec015b9430eaf1ec4f4eeb41cf6 P 8e16af0efe6c4f698e93d11250de344b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e16af0efe6c4f698e93d11250de344b, State: Initialized, Role: FOLLOWER
I20260812 21:04:03.642375 25520 consensus_queue.cc:260] T 46376ec015b9430eaf1ec4f4eeb41cf6 P 8e16af0efe6c4f698e93d11250de344b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "228262922b09413f8fe37401b0a932bd" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 46307 } } peers { permanent_uuid: "8e16af0efe6c4f698e93d11250de344b" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 35727 } } peers { permanent_uuid: "a9da4e76358f4ee18755f38b4d8de7ad" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 39443 } }
I20260812 21:04:03.642400 25525 tablet_bootstrap.cc:492] T 46376ec015b9430eaf1ec4f4eeb41cf6 P 228262922b09413f8fe37401b0a932bd: No bootstrap required, opened a new log
I20260812 21:04:03.642442 25525 ts_tablet_manager.cc:1397] T 46376ec015b9430eaf1ec4f4eeb41cf6 P 228262922b09413f8fe37401b0a932bd: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 21:04:03.642483 25479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:03.642411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:43007 (local address 127.11.254.130:60715)
0812 21:04:03.642444 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0812 21:04:03.642464 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.190:43007: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
I20260812 21:04:03.642539 25520 ts_tablet_manager.cc:1428] T 46376ec015b9430eaf1ec4f4eeb41cf6 P 8e16af0efe6c4f698e93d11250de344b: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 21:04:03.642570 25525 raft_consensus.cc:348] T 46376ec015b9430eaf1ec4f4eeb41cf6 P 228262922b09413f8fe37401b0a932bd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "228262922b09413f8fe37401b0a932bd" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 46307 } } peers { permanent_uuid: "8e16af0efe6c4f698e93d11250de344b" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 35727 } } peers { permanent_uuid: "a9da4e76358f4ee18755f38b4d8de7ad" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 39443 } }
I20260812 21:04:03.642634 25525 raft_consensus.cc:374] T 46376ec015b9430eaf1ec4f4eeb41cf6 P 228262922b09413f8fe37401b0a932bd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:04:03.642650 25525 raft_consensus.cc:729] T 46376ec015b9430eaf1ec4f4eeb41cf6 P 228262922b09413f8fe37401b0a932bd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 228262922b09413f8fe37401b0a932bd, State: Initialized, Role: FOLLOWER
I20260812 21:04:03.642710 25525 consensus_queue.cc:260] T 46376ec015b9430eaf1ec4f4eeb41cf6 P 228262922b09413f8fe37401b0a932bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "228262922b09413f8fe37401b0a932bd" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 46307 } } peers { permanent_uuid: "8e16af0efe6c4f698e93d11250de344b" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 35727 } } peers { permanent_uuid: "a9da4e76358f4ee18755f38b4d8de7ad" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 39443 } }
I20260812 21:04:03.642840 25480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:03.642744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:43007 (local address 127.11.254.129:40633)
0812 21:04:03.642784 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0812 21:04:03.642815 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.190:43007: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
I20260812 21:04:03.642835 25113 heartbeater.cc:503] Master 127.11.254.188:36857 was elected leader, sending a full tablet report...
I20260812 21:04:03.643044 25525 ts_tablet_manager.cc:1428] T 46376ec015b9430eaf1ec4f4eeb41cf6 P 228262922b09413f8fe37401b0a932bd: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 21:04:03.643457 25481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:03.643370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:43007 (local address 127.11.254.131:35967)
0812 21:04:03.643408 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0812 21:04:03.643431 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.190:43007: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260812 21:04:03.643523 24865 heartbeater.cc:503] Master 127.11.254.188:36857 was elected leader, sending a full tablet report...
I20260812 21:04:03.643743 25480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:03.643567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:43007 (local address 127.11.254.129:53905)
0812 21:04:03.643689 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0812 21:04:03.643716 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.190:43007: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105}
I20260812 21:04:03.643994 25338 heartbeater.cc:503] Master 127.11.254.188:36857 was elected leader, sending a full tablet report...
I20260812 21:04:03.644177 25481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:03.644091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:43007 (local address 127.11.254.131:44715)
0812 21:04:03.644128 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0812 21:04:03.644151 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.190:43007: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260812 21:04:03.743795 25534 raft_consensus.cc:482] T 46376ec015b9430eaf1ec4f4eeb41cf6 P 8e16af0efe6c4f698e93d11250de344b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 21:04:03.743914 25534 raft_consensus.cc:504] T 46376ec015b9430eaf1ec4f4eeb41cf6 P 8e16af0efe6c4f698e93d11250de344b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "228262922b09413f8fe37401b0a932bd" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 46307 } } peers { permanent_uuid: "8e16af0efe6c4f698e93d11250de344b" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 35727 } } peers { permanent_uuid: "a9da4e76358f4ee18755f38b4d8de7ad" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 39443 } }
I20260812 21:04:03.744104 25534 leader_election.cc:288] T 46376ec015b9430eaf1ec4f4eeb41cf6 P 8e16af0efe6c4f698e93d11250de344b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 228262922b09413f8fe37401b0a932bd (127.11.254.131:46307), a9da4e76358f4ee18755f38b4d8de7ad (127.11.254.130:39443)
I20260812 21:04:03.744343 25261 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "46376ec015b9430eaf1ec4f4eeb41cf6" candidate_uuid: "8e16af0efe6c4f698e93d11250de344b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "228262922b09413f8fe37401b0a932bd" is_pre_election: true
I20260812 21:04:03.744354 25036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "46376ec015b9430eaf1ec4f4eeb41cf6" candidate_uuid: "8e16af0efe6c4f698e93d11250de344b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9da4e76358f4ee18755f38b4d8de7ad" is_pre_election: true
I20260812 21:04:03.744434 25261 raft_consensus.cc:2454] T 46376ec015b9430eaf1ec4f4eeb41cf6 P 228262922b09413f8fe37401b0a932bd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e16af0efe6c4f698e93d11250de344b in term 0.
I20260812 21:04:03.744473 25036 raft_consensus.cc:2454] T 46376ec015b9430eaf1ec4f4eeb41cf6 P a9da4e76358f4ee18755f38b4d8de7ad [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e16af0efe6c4f698e93d11250de344b in term 0.
I20260812 21:04:03.744608 24692 leader_election.cc:302] T 46376ec015b9430eaf1ec4f4eeb41cf6 P 8e16af0efe6c4f698e93d11250de344b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 228262922b09413f8fe37401b0a932bd, 8e16af0efe6c4f698e93d11250de344b; no voters:
I20260812 21:04:03.744732 25534 raft_consensus.cc:2781] T 46376ec015b9430eaf1ec4f4eeb41cf6 P 8e16af0efe6c4f698e93d11250de344b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 21:04:03.744781 25534 raft_consensus.cc:482] T 46376ec015b9430eaf1ec4f4eeb41cf6 P 8e16af0efe6c4f698e93d11250de344b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 21:04:03.744794 25534 raft_consensus.cc:3037] T 46376ec015b9430eaf1ec4f4eeb41cf6 P 8e16af0efe6c4f698e93d11250de344b [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:04:03.745584 25534 raft_consensus.cc:504] T 46376ec015b9430eaf1ec4f4eeb41cf6 P 8e16af0efe6c4f698e93d11250de344b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "228262922b09413f8fe37401b0a932bd" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 46307 } } peers { permanent_uuid: "8e16af0efe6c4f698e93d11250de344b" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 35727 } } peers { permanent_uuid: "a9da4e76358f4ee18755f38b4d8de7ad" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 39443 } }
I20260812 21:04:03.745708 25534 leader_election.cc:288] T 46376ec015b9430eaf1ec4f4eeb41cf6 P 8e16af0efe6c4f698e93d11250de344b [CANDIDATE]: Term 1 election: Requested vote from peers 228262922b09413f8fe37401b0a932bd (127.11.254.131:46307), a9da4e76358f4ee18755f38b4d8de7ad (127.11.254.130:39443)
I20260812 21:04:03.745911 25261 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "46376ec015b9430eaf1ec4f4eeb41cf6" candidate_uuid: "8e16af0efe6c4f698e93d11250de344b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "228262922b09413f8fe37401b0a932bd"
I20260812 21:04:03.745906 25036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "46376ec015b9430eaf1ec4f4eeb41cf6" candidate_uuid: "8e16af0efe6c4f698e93d11250de344b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9da4e76358f4ee18755f38b4d8de7ad"
I20260812 21:04:03.746008 25261 raft_consensus.cc:3037] T 46376ec015b9430eaf1ec4f4eeb41cf6 P 228262922b09413f8fe37401b0a932bd [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:04:03.746016 25036 raft_consensus.cc:3037] T 46376ec015b9430eaf1ec4f4eeb41cf6 P a9da4e76358f4ee18755f38b4d8de7ad [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:04:03.746637 25036 raft_consensus.cc:2454] T 46376ec015b9430eaf1ec4f4eeb41cf6 P a9da4e76358f4ee18755f38b4d8de7ad [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e16af0efe6c4f698e93d11250de344b in term 1.
I20260812 21:04:03.746663 25261 raft_consensus.cc:2454] T 46376ec015b9430eaf1ec4f4eeb41cf6 P 228262922b09413f8fe37401b0a932bd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e16af0efe6c4f698e93d11250de344b in term 1.
I20260812 21:04:03.746814 24689 leader_election.cc:302] T 46376ec015b9430eaf1ec4f4eeb41cf6 P 8e16af0efe6c4f698e93d11250de344b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e16af0efe6c4f698e93d11250de344b, a9da4e76358f4ee18755f38b4d8de7ad; no voters:
I20260812 21:04:03.746937 25534 raft_consensus.cc:2781] T 46376ec015b9430eaf1ec4f4eeb41cf6 P 8e16af0efe6c4f698e93d11250de344b [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:04:03.747126 25534 raft_consensus.cc:686] T 46376ec015b9430eaf1ec4f4eeb41cf6 P 8e16af0efe6c4f698e93d11250de344b [term 1 LEADER]: Becoming Leader. State: Replica: 8e16af0efe6c4f698e93d11250de344b, State: Running, Role: LEADER
I20260812 21:04:03.747184 25534 consensus_queue.cc:237] T 46376ec015b9430eaf1ec4f4eeb41cf6 P 8e16af0efe6c4f698e93d11250de344b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "228262922b09413f8fe37401b0a932bd" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 46307 } } peers { permanent_uuid: "8e16af0efe6c4f698e93d11250de344b" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 35727 } } peers { permanent_uuid: "a9da4e76358f4ee18755f38b4d8de7ad" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 39443 } }
I20260812 21:04:03.747498 25480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:03.747378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:43007 (local address 127.11.254.129:38945)
0812 21:04:03.747424 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0812 21:04:03.747464 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.190:43007: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260812 21:04:03.747795 24580 catalog_manager.cc:5495] T 46376ec015b9430eaf1ec4f4eeb41cf6 P 8e16af0efe6c4f698e93d11250de344b reported cstate change: term changed from 0 to 1, leader changed from <none> to 8e16af0efe6c4f698e93d11250de344b (127.11.254.129). New cstate: current_term: 1 leader_uuid: "8e16af0efe6c4f698e93d11250de344b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "228262922b09413f8fe37401b0a932bd" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 46307 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8e16af0efe6c4f698e93d11250de344b" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 35727 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a9da4e76358f4ee18755f38b4d8de7ad" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 39443 } health_report { overall_health: UNKNOWN } } }
I20260812 21:04:03.754870 25516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:03.752937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49114 (local address 127.11.254.129:35727)
0812 21:04:03.752978 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0812 21:04:03.752980 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:04:03.752989 (+ 9us) server_negotiation.cc:407] Connection header received
0812 21:04:03.753057 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:04:03.753058 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:04:03.753066 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:04:03.753110 (+ 44us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 21:04:03.753196 (+ 86us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:03.753662 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:03.754368 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:03.754559 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:03.754607 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:03.754705 (+ 98us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 21:04:03.754729 (+ 24us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 21:04:03.754760 (+ 31us) server_negotiation.cc:1035] Waiting for connection context
0812 21:04:03.754796 (+ 36us) server_negotiation.cc:299] Negotiation successful
0812 21:04:03.754813 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 21:04:03.755689 25261 raft_consensus.cc:1261] T 46376ec015b9430eaf1ec4f4eeb41cf6 P 228262922b09413f8fe37401b0a932bd [term 1 FOLLOWER]: Refusing update from remote peer 8e16af0efe6c4f698e93d11250de344b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 21:04:03.755689 25036 raft_consensus.cc:1261] T 46376ec015b9430eaf1ec4f4eeb41cf6 P a9da4e76358f4ee18755f38b4d8de7ad [term 1 FOLLOWER]: Refusing update from remote peer 8e16af0efe6c4f698e93d11250de344b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 21:04:03.755904 25537 consensus_queue.cc:1035] T 46376ec015b9430eaf1ec4f4eeb41cf6 P 8e16af0efe6c4f698e93d11250de344b [LEADER]: Connected to new peer: Peer: permanent_uuid: "228262922b09413f8fe37401b0a932bd" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 46307 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 21:04:03.756016 25534 consensus_queue.cc:1035] T 46376ec015b9430eaf1ec4f4eeb41cf6 P 8e16af0efe6c4f698e93d11250de344b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9da4e76358f4ee18755f38b4d8de7ad" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 39443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 21:04:03.756664 25479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:03.756536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:43007 (local address 127.11.254.130:47837)
0812 21:04:03.756595 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0812 21:04:03.756632 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.190:43007: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260812 21:04:03.756850 25481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:03.756716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:43007 (local address 127.11.254.131:39551)
0812 21:04:03.756789 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0812 21:04:03.756822 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.190:43007: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260812 21:04:03.757993 25480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:03.757827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:43007 (local address 127.11.254.129:35305)
0812 21:04:03.757900 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0812 21:04:03.757939 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.190:43007: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260812 21:04:03.758543 25481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:03.758327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:43007 (local address 127.11.254.131:48793)
0812 21:04:03.758483 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0812 21:04:03.758513 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.190:43007: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140}
I20260812 21:04:03.758836 25549 mvcc.cc:205] Tried to move back new op lower bound from 7317785164821561344 to 7317785164788838400. Current Snapshot: MvccSnapshot[applied={T|T < 7317785164821561344}]
I20260812 21:04:03.758898 25547 mvcc.cc:205] Tried to move back new op lower bound from 7317785164821561344 to 7317785164788838400. Current Snapshot: MvccSnapshot[applied={T|T < 7317785164821561344 or (T in {7317785164821561344})}]
I20260812 21:04:03.758927 25479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:03.758843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:43007 (local address 127.11.254.130:57929)
0812 21:04:03.758879 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0812 21:04:03.758901 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.190:43007: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
I20260812 21:04:03.758987 24580 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32998:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 21:04:03.759071 24580 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32998:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 21:04:03.760304 24580 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7e4700f28485420fbba29f2e545c71df: Sending DeleteTablet for 3 replicas of tablet 46376ec015b9430eaf1ec4f4eeb41cf6
I20260812 21:04:03.760596 24760 tablet_service.cc:1514] Processing DeleteTablet for tablet 46376ec015b9430eaf1ec4f4eeb41cf6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:04:03 UTC) from {username='oryx', principal='oryx/127.11.254.188@KRBTEST.COM'} at 127.0.0.1:49102
I20260812 21:04:03.760643 25235 tablet_service.cc:1514] Processing DeleteTablet for tablet 46376ec015b9430eaf1ec4f4eeb41cf6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:04:03 UTC) from {username='oryx', principal='oryx/127.11.254.188@KRBTEST.COM'} at 127.0.0.1:41234
I20260812 21:04:03.760783 25563 tablet_replica.cc:330] T 46376ec015b9430eaf1ec4f4eeb41cf6 P 8e16af0efe6c4f698e93d11250de344b: stopping tablet replica
I20260812 21:04:03.760787 25000 tablet_service.cc:1514] Processing DeleteTablet for tablet 46376ec015b9430eaf1ec4f4eeb41cf6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:04:03 UTC) from {username='oryx', principal='oryx/127.11.254.188@KRBTEST.COM'} at 127.0.0.1:57260
I20260812 21:04:03.760856 25564 tablet_replica.cc:330] T 46376ec015b9430eaf1ec4f4eeb41cf6 P 228262922b09413f8fe37401b0a932bd: stopping tablet replica
I20260812 21:04:03.760913 25564 raft_consensus.cc:2229] T 46376ec015b9430eaf1ec4f4eeb41cf6 P 228262922b09413f8fe37401b0a932bd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 21:04:03.760972 25564 raft_consensus.cc:2258] T 46376ec015b9430eaf1ec4f4eeb41cf6 P 228262922b09413f8fe37401b0a932bd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 21:04:03.760939 25563 raft_consensus.cc:2229] T 46376ec015b9430eaf1ec4f4eeb41cf6 P 8e16af0efe6c4f698e93d11250de344b [term 1 LEADER]: Raft consensus shutting down.
I20260812 21:04:03.761116 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 24509
I20260812 21:04:03.761932 25564 ts_tablet_manager.cc:1905] T 46376ec015b9430eaf1ec4f4eeb41cf6 P 228262922b09413f8fe37401b0a932bd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 21:04:03.764385 25564 ts_tablet_manager.cc:1918] T 46376ec015b9430eaf1ec4f4eeb41cf6 P 228262922b09413f8fe37401b0a932bd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 21:04:03.764444 25564 log.cc:1196] T 46376ec015b9430eaf1ec4f4eeb41cf6 P 228262922b09413f8fe37401b0a932bd: Deleting WAL directory at /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/ts-2/wal/wals/46376ec015b9430eaf1ec4f4eeb41cf6
I20260812 21:04:03.764686 25564 ts_tablet_manager.cc:1939] T 46376ec015b9430eaf1ec4f4eeb41cf6 P 228262922b09413f8fe37401b0a932bd: Deleting consensus metadata
I20260812 21:04:03.765076 24563 catalog_manager.cc:4841] TS 228262922b09413f8fe37401b0a932bd (127.11.254.131:46307): tablet 46376ec015b9430eaf1ec4f4eeb41cf6 (table test-table [id=e292eabdcd264763bf14981d26550474]) successfully deleted
I20260812 21:04:03.776227 25566 tablet_replica.cc:330] T 46376ec015b9430eaf1ec4f4eeb41cf6 P a9da4e76358f4ee18755f38b4d8de7ad: stopping tablet replica
I20260812 21:04:03.776331 25566 raft_consensus.cc:2229] T 46376ec015b9430eaf1ec4f4eeb41cf6 P a9da4e76358f4ee18755f38b4d8de7ad [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 21:04:03.776393 25566 raft_consensus.cc:2258] T 46376ec015b9430eaf1ec4f4eeb41cf6 P a9da4e76358f4ee18755f38b4d8de7ad [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 21:04:03.776726 25566 ts_tablet_manager.cc:1905] T 46376ec015b9430eaf1ec4f4eeb41cf6 P a9da4e76358f4ee18755f38b4d8de7ad: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 21:04:03.778470 25566 ts_tablet_manager.cc:1918] T 46376ec015b9430eaf1ec4f4eeb41cf6 P a9da4e76358f4ee18755f38b4d8de7ad: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 21:04:03.778550 25566 log.cc:1196] T 46376ec015b9430eaf1ec4f4eeb41cf6 P a9da4e76358f4ee18755f38b4d8de7ad: Deleting WAL directory at /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786568607354921-12282-0/minicluster-data/ts-1/wal/wals/46376ec015b9430eaf1ec4f4eeb41cf6
I20260812 21:04:03.778784 25566 ts_tablet_manager.cc:1939] T 46376ec015b9430eaf1ec4f4eeb41cf6 P a9da4e76358f4ee18755f38b4d8de7ad: Deleting consensus metadata
I20260812 21:04:03.779160 24560 catalog_manager.cc:4841] TS a9da4e76358f4ee18755f38b4d8de7ad (127.11.254.130:39443): tablet 46376ec015b9430eaf1ec4f4eeb41cf6 (table test-table [id=e292eabdcd264763bf14981d26550474]) successfully deleted
W20260812 21:04:03.782310 25489 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:04:03.782162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.129:35727 (local address 127.0.0.1:49118)
0812 21:04:03.782224 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0812 21:04:03.782254 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.129:35727: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":35}
W20260812 21:04:03.782480 24561 catalog_manager.cc:4584] TS 8e16af0efe6c4f698e93d11250de344b (127.11.254.129:35727): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 46376ec015b9430eaf1ec4f4eeb41cf6: Network error: Client connection negotiation failed: client connection to 127.11.254.129:35727: connect: Connection reset by peer (error 104)
I20260812 21:04:03.782521 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 24880
I20260812 21:04:03.798539 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 25121
I20260812 21:04:03.808583 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 24025
I20260812 21:04:03.819344 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 24353
2026-08-12T21:04:03Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (6799 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568607354921-12282-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 21:04:03 dist-test-slave-vpvm krb5kdc[25571](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 21:04:03 dist-test-slave-vpvm krb5kdc[25571](info): set up 2 sockets
Aug 12 21:04:03 dist-test-slave-vpvm krb5kdc[25571](info): commencing operation
krb5kdc: starting...
W20260812 21:04:05.016726 12282 mini_kdc.cc:121] Time spent starting KDC: real 1.168s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568607354921-12282-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568607354921-12282-0/krb5kdc/test-user.keytab.
Aug 12 21:04:05 dist-test-slave-vpvm krb5kdc[25571](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568645, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T21:04:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T21:04:05Z Disabled control of system clock
WARNING: no policy specified for oryx/127.11.254.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.254.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.254.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568607354921-12282-0/krb5kdc/oryx_127.11.254.190.keytab.
Entry for principal oryx/127.11.254.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568607354921-12282-0/krb5kdc/oryx_127.11.254.190.keytab.
I20260812 21:04:05.142792 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568607354921-12282-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568607354921-12282-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568607354921-12282-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.254.190:43839
--webserver_interface=127.11.254.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568607354921-12282-0/krb5kdc/oryx_127.11.254.190.keytab
--principal=oryx/127.11.254.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.254.148:35883
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.254.190:43839
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568607354921-12282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568607354921-12282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568607354921-12282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:04:05.226244 25590 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:04:05.226472 25590 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:04:05.226495 25590 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:04:05.227691 25590 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 21:04:05.227735 25590 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260812 21:04:05.227747 25590 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:04:05.227759 25590 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:04:05.227775 25590 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 21:04:05.227799 25590 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 21:04:05.229370 25590 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.254.148:35883
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568607354921-12282-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568607354921-12282-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.254.190:43839
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568607354921-12282-0/krb5kdc/oryx_127.11.254.190.keytab
--principal=oryx/127.11.254.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.254.190:43839
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.254.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25590
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568607354921-12282-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:04:05.229640 25590 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:04:05.229863 25590 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:04:05.235051 25597 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:04:05.235045 25600 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:04:05.235051 25596 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:04:05.235350 25590 server_base.cc:1034] running on GCE node
I20260812 21:04:05.235636 25590 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:04:05.235931 25590 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:04:05.237118 25590 hybrid_clock.cc:648] HybridClock initialized: now 1786568645237100 us; error 33 us; skew 500 ppm
Aug 12 21:04:05 dist-test-slave-vpvm krb5kdc[25571](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568645, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.254.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:04:05.240120 25590 init.cc:377] Logged in from keytab as oryx/127.11.254.190@KRBTEST.COM (short username oryx)
I20260812 21:04:05.240630 25590 webserver.cc:460] Webserver started at http://127.11.254.190:40081/ using document root <none> and password file <none>
I20260812 21:04:05.240824 25590 fs_manager.cc:359] Metadata directory not provided
I20260812 21:04:05.240865 25590 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:04:05.240957 25590 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:04:05.242079 25590 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568607354921-12282-0/minicluster-data/master-0/data/instance:
uuid: "5eb7e77e033b4ec48c7dabe9b94175eb"
format_stamp: "Formatted at 2026-08-12 21:04:05 on dist-test-slave-vpvm"
server_key: "d118c5cb0bcc16b3c1eba324c74a69ac"
server_key_iv: "e570ad4dd0bec607895324d5f9bbfa1c"
server_key_version: "encryptionkey@0"
I20260812 21:04:05.242470 25590 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568607354921-12282-0/minicluster-data/master-0/wal/instance:
uuid: "5eb7e77e033b4ec48c7dabe9b94175eb"
format_stamp: "Formatted at 2026-08-12 21:04:05 on dist-test-slave-vpvm"
server_key: "d118c5cb0bcc16b3c1eba324c74a69ac"
server_key_iv: "e570ad4dd0bec607895324d5f9bbfa1c"
server_key_version: "encryptionkey@0"
I20260812 21:04:05.243815 25590 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260812 21:04:05.244495 25610 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:04:05.244661 25590 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 21:04:05.244753 25590 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568607354921-12282-0/minicluster-data/master-0/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568607354921-12282-0/minicluster-data/master-0/wal
uuid: "5eb7e77e033b4ec48c7dabe9b94175eb"
format_stamp: "Formatted at 2026-08-12 21:04:05 on dist-test-slave-vpvm"
server_key: "d118c5cb0bcc16b3c1eba324c74a69ac"
server_key_iv: "e570ad4dd0bec607895324d5f9bbfa1c"
server_key_version: "encryptionkey@0"
I20260812 21:04:05.244817 25590 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568607354921-12282-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568607354921-12282-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568607354921-12282-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:04:05.254786 25590 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:04:05.256577 25590 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:04:05.256718 25590 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:04:05.261399 25590 rpc_server.cc:307] RPC server started. Bound to: 127.11.254.190:43839
I20260812 21:04:05.261448 25713 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.254.190:43839 every 8 connection(s)
I20260812 21:04:05.261788 25590 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
I20260812 21:04:05.262497 25714 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:04:05.264957 25714 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5eb7e77e033b4ec48c7dabe9b94175eb: Bootstrap starting.
I20260812 21:04:05.265595 25714 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5eb7e77e033b4ec48c7dabe9b94175eb: Neither blocks nor log segments found. Creating new log.
I20260812 21:04:05.265879 25714 log.cc:824] T 00000000000000000000000000000000 P 5eb7e77e033b4ec48c7dabe9b94175eb: Log is configured to *not* fsync() on all Append() calls
I20260812 21:04:05.266594 25714 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5eb7e77e033b4ec48c7dabe9b94175eb: No bootstrap required, opened a new log
I20260812 21:04:05.268538 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 25590
I20260812 21:04:05.268656 12282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568607354921-12282-0/minicluster-data/master-0/wal/instance
I20260812 21:04:05.268858 12282 external_mini_cluster.cc:1428] Setting key fb32efe121e63c99ebc1890eed604386
I20260812 21:04:05.268865 25714 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5eb7e77e033b4ec48c7dabe9b94175eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5eb7e77e033b4ec48c7dabe9b94175eb" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 43839 } }
I20260812 21:04:05.269044 25714 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5eb7e77e033b4ec48c7dabe9b94175eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:04:05.269107 25714 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5eb7e77e033b4ec48c7dabe9b94175eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5eb7e77e033b4ec48c7dabe9b94175eb, State: Initialized, Role: FOLLOWER
I20260812 21:04:05.269272 25714 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5eb7e77e033b4ec48c7dabe9b94175eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5eb7e77e033b4ec48c7dabe9b94175eb" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 43839 } }
I20260812 21:04:05.269372 25714 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5eb7e77e033b4ec48c7dabe9b94175eb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 21:04:05.269424 25714 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5eb7e77e033b4ec48c7dabe9b94175eb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 21:04:05.269487 25714 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5eb7e77e033b4ec48c7dabe9b94175eb [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:04:05.270244 25714 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5eb7e77e033b4ec48c7dabe9b94175eb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5eb7e77e033b4ec48c7dabe9b94175eb" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 43839 } }
I20260812 21:04:05.270340 25714 leader_election.cc:302] T 00000000000000000000000000000000 P 5eb7e77e033b4ec48c7dabe9b94175eb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5eb7e77e033b4ec48c7dabe9b94175eb; no voters:
I20260812 21:04:05.270516 25714 leader_election.cc:288] T 00000000000000000000000000000000 P 5eb7e77e033b4ec48c7dabe9b94175eb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 21:04:05.270567 25725 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5eb7e77e033b4ec48c7dabe9b94175eb [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:04:05.270704 25725 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5eb7e77e033b4ec48c7dabe9b94175eb [term 1 LEADER]: Becoming Leader. State: Replica: 5eb7e77e033b4ec48c7dabe9b94175eb, State: Running, Role: LEADER
I20260812 21:04:05.270807 25714 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5eb7e77e033b4ec48c7dabe9b94175eb [sys.catalog]: configured and running, proceeding with master startup.
I20260812 21:04:05.270841 25725 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5eb7e77e033b4ec48c7dabe9b94175eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5eb7e77e033b4ec48c7dabe9b94175eb" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 43839 } }
I20260812 21:04:05.271274 25726 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5eb7e77e033b4ec48c7dabe9b94175eb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5eb7e77e033b4ec48c7dabe9b94175eb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5eb7e77e033b4ec48c7dabe9b94175eb" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 43839 } } }
I20260812 21:04:05.271366 25726 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5eb7e77e033b4ec48c7dabe9b94175eb [sys.catalog]: This master's current role is: LEADER
I20260812 21:04:05.271505 25727 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5eb7e77e033b4ec48c7dabe9b94175eb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5eb7e77e033b4ec48c7dabe9b94175eb. Latest consensus state: current_term: 1 leader_uuid: "5eb7e77e033b4ec48c7dabe9b94175eb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5eb7e77e033b4ec48c7dabe9b94175eb" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 43839 } } }
I20260812 21:04:05.271575 25727 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5eb7e77e033b4ec48c7dabe9b94175eb [sys.catalog]: This master's current role is: LEADER
W20260812 21:04:05.272688 25751 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5eb7e77e033b4ec48c7dabe9b94175eb: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 21:04:05.272737 25751 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 21:04:05.272786 25739 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 21:04:05.272921 25739 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 21:04:05.274513 25739 catalog_manager.cc:1347] Generated new cluster ID: 1d1894d99e6641a2b9a26e566faef717
I20260812 21:04:05.274572 25739 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 21:04:05 dist-test-slave-vpvm krb5kdc[25571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568645, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.11.254.190@KRBTEST.COM
I20260812 21:04:05.280581 25724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:05.269409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58258 (local address 127.11.254.190:43839)
0812 21:04:05.269641 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0812 21:04:05.269651 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 21:04:05.269683 (+ 32us) server_negotiation.cc:407] Connection header received
0812 21:04:05.269790 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:04:05.269796 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:04:05.269839 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:04:05.269972 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:04:05.270737 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:05.272131 (+ 1394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:05.273125 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:05.273714 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:05.273828 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:05.276829 (+ 3001us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:04:05.276833 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:04:05.276833 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:04:05.276846 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:04:05.278858 (+ 2012us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:04:05.279289 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:04:05.279292 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:04:05.279292 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:04:05.279406 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:04:05.279608 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:04:05.279609 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:04:05.279609 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:04:05.279911 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:04:05.280067 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0812 21:04:05.280191 (+ 124us) server_negotiation.cc:299] Negotiation successful
0812 21:04:05.280353 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":171,"thread_start_us":96,"threads_started":1}
I20260812 21:04:05.281736 25739 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 21:04:05.282298 25739 catalog_manager.cc:1504] Loading token signing keys...
I20260812 21:04:05.290103 25739 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5eb7e77e033b4ec48c7dabe9b94175eb: Generated new TSK 0
I20260812 21:04:05.290287 25739 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.11.254.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.254.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.254.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568607354921-12282-0/krb5kdc/oryx_127.11.254.129.keytab.
Entry for principal oryx/127.11.254.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568607354921-12282-0/krb5kdc/oryx_127.11.254.129.keytab.
I20260812 21:04:05.323350 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568607354921-12282-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568607354921-12282-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568607354921-12282-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568607354921-12282-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.254.129:0
--local_ip_for_outbound_sockets=127.11.254.129
--webserver_interface=127.11.254.129
--webserver_port=0
--tserver_master_addrs=127.11.254.190:43839
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568607354921-12282-0/krb5kdc/oryx_127.11.254.129.keytab
--principal=oryx/127.11.254.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.254.148:35883
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568607354921-12282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568607354921-12282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568607354921-12282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:04:05.407153 25762 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:04:05.407369 25762 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:04:05.407402 25762 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:04:05.408556 25762 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:04:05.408600 25762 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:04:05.408627 25762 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 21:04:05.408650 25762 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.254.129
I20260812 21:04:05.410161 25762 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.254.148:35883
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568607354921-12282-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568607354921-12282-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568607354921-12282-0/krb5kdc/oryx_127.11.254.129.keytab
--principal=oryx/127.11.254.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.254.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568607354921-12282-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.254.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.11.254.190:43839
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25762
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.254.129
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568607354921-12282-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:04:05.410403 25762 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:04:05.410622 25762 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:04:05.415613 25769 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:04:05.415649 25768 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:04:05.415591 25773 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:04:05.415838 25762 server_base.cc:1034] running on GCE node
I20260812 21:04:05.416060 25762 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:04:05.416297 25762 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:04:05.417440 25762 hybrid_clock.cc:648] HybridClock initialized: now 1786568645417429 us; error 32 us; skew 500 ppm
Aug 12 21:04:05 dist-test-slave-vpvm krb5kdc[25571](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568645, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.254.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:04:05.419993 25762 init.cc:377] Logged in from keytab as oryx/127.11.254.129@KRBTEST.COM (short username oryx)
I20260812 21:04:05.420423 25762 webserver.cc:460] Webserver started at http://127.11.254.129:32887/ using document root <none> and password file <none>
I20260812 21:04:05.420629 25762 fs_manager.cc:359] Metadata directory not provided
I20260812 21:04:05.420677 25762 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:04:05.420807 25762 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:04:05.422036 25762 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568607354921-12282-0/minicluster-data/ts-0/data/instance:
uuid: "0d68d6ebf5ac47cfa96cfe32cc083243"
format_stamp: "Formatted at 2026-08-12 21:04:05 on dist-test-slave-vpvm"
server_key: "a292f2085e60752b97b558e87143255d"
server_key_iv: "f7ef97eb948ad05dc71ca245f1021d94"
server_key_version: "encryptionkey@0"
I20260812 21:04:05.422425 25762 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568607354921-12282-0/minicluster-data/ts-0/wal/instance:
uuid: "0d68d6ebf5ac47cfa96cfe32cc083243"
format_stamp: "Formatted at 2026-08-12 21:04:05 on dist-test-slave-vpvm"
server_key: "a292f2085e60752b97b558e87143255d"
server_key_iv: "f7ef97eb948ad05dc71ca245f1021d94"
server_key_version: "encryptionkey@0"
I20260812 21:04:05.423736 25762 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 21:04:05.424482 25791 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:04:05.424670 25762 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 21:04:05.424755 25762 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568607354921-12282-0/minicluster-data/ts-0/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568607354921-12282-0/minicluster-data/ts-0/wal
uuid: "0d68d6ebf5ac47cfa96cfe32cc083243"
format_stamp: "Formatted at 2026-08-12 21:04:05 on dist-test-slave-vpvm"
server_key: "a292f2085e60752b97b558e87143255d"
server_key_iv: "f7ef97eb948ad05dc71ca245f1021d94"
server_key_version: "encryptionkey@0"
I20260812 21:04:05.424824 25762 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568607354921-12282-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568607354921-12282-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568607354921-12282-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:04:05.436796 25762 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:04:05.438588 25762 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:04:05.438723 25762 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:04:05.439235 25762 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:04:05.439278 25762 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:04:05.439311 25762 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:04:05.439334 25762 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:04:05.446017 25762 rpc_server.cc:307] RPC server started. Bound to: 127.11.254.129:36765
I20260812 21:04:05.446053 25983 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.254.129:36765 every 8 connection(s)
Aug 12 21:04:05 dist-test-slave-vpvm krb5kdc[25571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568645, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.254.129@KRBTEST.COM for oryx/127.11.254.190@KRBTEST.COM
I20260812 21:04:05.446409 25762 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568607354921-12282-0/minicluster-data/ts-0/data/info.pb
I20260812 21:04:05.449385 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 25762
I20260812 21:04:05.449537 12282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568607354921-12282-0/minicluster-data/ts-0/wal/instance
I20260812 21:04:05.449739 12282 external_mini_cluster.cc:1428] Setting key 88b8d822744a5f01bd9f72c25b690f77
I20260812 21:04:05.450027 25724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:05.439737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.129:36519 (local address 127.11.254.190:43839)
0812 21:04:05.439792 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0812 21:04:05.439794 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:04:05.439978 (+ 184us) server_negotiation.cc:407] Connection header received
0812 21:04:05.440511 (+ 533us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:04:05.440514 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:04:05.440531 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:04:05.440584 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:04:05.441406 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:05.442215 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:05.443797 (+ 1582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:05.444126 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:05.444188 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:05.446976 (+ 2788us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:04:05.446978 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:04:05.446978 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:04:05.446985 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:04:05.448534 (+ 1549us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:04:05.449005 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:04:05.449006 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:04:05.449006 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:04:05.449066 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:04:05.449307 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:04:05.449308 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:04:05.449309 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:04:05.449444 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:04:05.449508 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0812 21:04:05.449810 (+ 302us) server_negotiation.cc:299] Negotiation successful
0812 21:04:05.449909 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260812 21:04:05.450201 25809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:05.439704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:43839 (local address 127.11.254.129:36519)
0812 21:04:05.439863 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0812 21:04:05.439901 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 21:04:05.440411 (+ 510us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:04:05.440656 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:04:05.440661 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:04:05.441004 (+ 343us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:04:05.441315 (+ 311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:05.441322 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:05.442307 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:05.442309 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:05.443692 (+ 1383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:05.443697 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:05.444224 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:05.444225 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:05.444386 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:05.444989 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:04:05.445001 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:04:05.446869 (+ 1868us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:04:05.448631 (+ 1762us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:04:05.448638 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:04:05.448640 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:04:05.448935 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:04:05.449129 (+ 194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:04:05.449129 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:04:05.449130 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:04:05.449235 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:04:05.449506 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:04:05.449514 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:04:05.449669 (+ 155us) client_negotiation.cc:769] Sending connection context
0812 21:04:05.449776 (+ 107us) client_negotiation.cc:240] Negotiation successful
0812 21:04:05.449975 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":79,"threads_started":1}
I20260812 21:04:05.450695 25985 heartbeater.cc:347] Connected to a master server at 127.11.254.190:43839
I20260812 21:04:05.450781 25985 heartbeater.cc:464] Registering TS with master...
I20260812 21:04:05.450978 25985 heartbeater.cc:511] Master 127.11.254.190:43839 requested a full tablet report, sending...
I20260812 21:04:05.451560 25632 ts_manager.cc:194] Registered new tserver with Master: 0d68d6ebf5ac47cfa96cfe32cc083243 (127.11.254.129:36765)
I20260812 21:04:05.452417 25632 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.11.254.129@KRBTEST.COM'} at 127.11.254.129:36519
I20260812 21:04:05.457231 25724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:05.451949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.129:47457 (local address 127.11.254.190:43839)
0812 21:04:05.451996 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0812 21:04:05.451997 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 21:04:05.452004 (+ 7us) server_negotiation.cc:407] Connection header received
0812 21:04:05.452024 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:04:05.452025 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:04:05.452033 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:04:05.452090 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:04:05.452446 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:05.453053 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:05.453746 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:05.454042 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:05.454111 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:05.454953 (+ 842us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:04:05.454954 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:04:05.454955 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:04:05.454962 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:04:05.456247 (+ 1285us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:04:05.456612 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:04:05.456612 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:04:05.456613 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:04:05.456656 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:04:05.456831 (+ 175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:04:05.456831 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:04:05.456832 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:04:05.456919 (+ 87us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:04:05.456961 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0812 21:04:05.457112 (+ 151us) server_negotiation.cc:299] Negotiation successful
0812 21:04:05.457170 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260812 21:04:05.457307 25992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:05.451711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:43839 (local address 127.11.254.129:47457)
0812 21:04:05.451861 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0812 21:04:05.451873 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 21:04:05.451925 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:04:05.452098 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:04:05.452099 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:04:05.452290 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:04:05.452354 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:05.452356 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:05.453145 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:05.453146 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:05.453677 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:05.453678 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:05.454109 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:05.454109 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:05.454162 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:05.454571 (+ 409us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:04:05.454576 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:04:05.454886 (+ 310us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:04:05.456327 (+ 1441us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:04:05.456328 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:04:05.456329 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:04:05.456543 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:04:05.456709 (+ 166us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:04:05.456710 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:04:05.456710 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:04:05.456779 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:04:05.456969 (+ 190us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:04:05.456970 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:04:05.457023 (+ 53us) client_negotiation.cc:769] Sending connection context
0812 21:04:05.457095 (+ 72us) client_negotiation.cc:240] Negotiation successful
0812 21:04:05.457212 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"mutex_wait_us":7,"thread_start_us":84,"threads_started":1}
WARNING: no policy specified for oryx/127.11.254.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.254.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.254.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568607354921-12282-0/krb5kdc/oryx_127.11.254.130.keytab.
Entry for principal oryx/127.11.254.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568607354921-12282-0/krb5kdc/oryx_127.11.254.130.keytab.
I20260812 21:04:05.477842 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568607354921-12282-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568607354921-12282-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568607354921-12282-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568607354921-12282-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.254.130:0
--local_ip_for_outbound_sockets=127.11.254.130
--webserver_interface=127.11.254.130
--webserver_port=0
--tserver_master_addrs=127.11.254.190:43839
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568607354921-12282-0/krb5kdc/oryx_127.11.254.130.keytab
--principal=oryx/127.11.254.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.254.148:35883
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568607354921-12282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568607354921-12282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568607354921-12282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:04:05.560318 25998 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:04:05.560540 25998 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:04:05.560565 25998 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:04:05.561712 25998 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:04:05.561750 25998 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:04:05.561769 25998 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 21:04:05.561784 25998 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.254.130
I20260812 21:04:05.563396 25998 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.254.148:35883
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568607354921-12282-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568607354921-12282-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568607354921-12282-0/krb5kdc/oryx_127.11.254.130.keytab
--principal=oryx/127.11.254.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.254.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568607354921-12282-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.254.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.11.254.190:43839
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25998
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.254.130
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568607354921-12282-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:04:05.563617 25998 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:04:05.563818 25998 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:04:05.569121 26005 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:04:05.569134 26004 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:04:05.569134 26007 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:04:05.569536 25998 server_base.cc:1034] running on GCE node
I20260812 21:04:05.569741 25998 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:04:05.569984 25998 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:04:05.571141 25998 hybrid_clock.cc:648] HybridClock initialized: now 1786568645571123 us; error 34 us; skew 500 ppm
Aug 12 21:04:05 dist-test-slave-vpvm krb5kdc[25571](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568645, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.254.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:04:05.574087 25998 init.cc:377] Logged in from keytab as oryx/127.11.254.130@KRBTEST.COM (short username oryx)
I20260812 21:04:05.574543 25998 webserver.cc:460] Webserver started at http://127.11.254.130:34549/ using document root <none> and password file <none>
I20260812 21:04:05.574783 25998 fs_manager.cc:359] Metadata directory not provided
I20260812 21:04:05.574833 25998 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:04:05.574960 25998 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:04:05.576016 25998 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568607354921-12282-0/minicluster-data/ts-1/data/instance:
uuid: "868e98cd6025463aa10e6b78dff569a7"
format_stamp: "Formatted at 2026-08-12 21:04:05 on dist-test-slave-vpvm"
server_key: "51dacbda5953ec12f6a0f1198ca0389d"
server_key_iv: "7bcb5d38d9eeb3224372484b99088e0e"
server_key_version: "encryptionkey@0"
I20260812 21:04:05.576360 25998 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568607354921-12282-0/minicluster-data/ts-1/wal/instance:
uuid: "868e98cd6025463aa10e6b78dff569a7"
format_stamp: "Formatted at 2026-08-12 21:04:05 on dist-test-slave-vpvm"
server_key: "51dacbda5953ec12f6a0f1198ca0389d"
server_key_iv: "7bcb5d38d9eeb3224372484b99088e0e"
server_key_version: "encryptionkey@0"
I20260812 21:04:05.577656 25998 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 21:04:05.578413 26018 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:04:05.578564 25998 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 21:04:05.578686 25998 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568607354921-12282-0/minicluster-data/ts-1/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568607354921-12282-0/minicluster-data/ts-1/wal
uuid: "868e98cd6025463aa10e6b78dff569a7"
format_stamp: "Formatted at 2026-08-12 21:04:05 on dist-test-slave-vpvm"
server_key: "51dacbda5953ec12f6a0f1198ca0389d"
server_key_iv: "7bcb5d38d9eeb3224372484b99088e0e"
server_key_version: "encryptionkey@0"
I20260812 21:04:05.578754 25998 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568607354921-12282-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568607354921-12282-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568607354921-12282-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:04:05.629578 25998 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:04:05.631521 25998 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:04:05.631683 25998 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:04:05.632232 25998 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:04:05.632275 25998 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:04:05.632305 25998 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:04:05.632321 25998 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:04:05.638929 25998 rpc_server.cc:307] RPC server started. Bound to: 127.11.254.130:35013
I20260812 21:04:05.639016 26196 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.254.130:35013 every 8 connection(s)
Aug 12 21:04:05 dist-test-slave-vpvm krb5kdc[25571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568645, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.254.130@KRBTEST.COM for oryx/127.11.254.190@KRBTEST.COM
I20260812 21:04:05.639315 25998 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568607354921-12282-0/minicluster-data/ts-1/data/info.pb
I20260812 21:04:05.642361 25724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:05.632677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.130:49787 (local address 127.11.254.190:43839)
0812 21:04:05.632765 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0812 21:04:05.632767 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:04:05.632958 (+ 191us) server_negotiation.cc:407] Connection header received
0812 21:04:05.633467 (+ 509us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:04:05.633468 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:04:05.633482 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:04:05.633559 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:04:05.634418 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:05.635203 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:05.636730 (+ 1527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:05.637029 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:05.637108 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:05.639775 (+ 2667us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:04:05.639777 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:04:05.639777 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:04:05.639786 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:04:05.641034 (+ 1248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:04:05.641415 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:04:05.641416 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:04:05.641416 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:04:05.641474 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:04:05.641701 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:04:05.641702 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:04:05.641703 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:04:05.641818 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:04:05.641876 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0812 21:04:05.642182 (+ 306us) server_negotiation.cc:299] Negotiation successful
0812 21:04:05.642270 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260812 21:04:05.642521 26033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:05.632646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:43839 (local address 127.11.254.130:49787)
0812 21:04:05.632828 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0812 21:04:05.632872 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 21:04:05.633368 (+ 496us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:04:05.633625 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:04:05.633629 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:04:05.634025 (+ 396us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:04:05.634307 (+ 282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:05.634314 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:05.635305 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:05.635306 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:05.636638 (+ 1332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:05.636643 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:05.637119 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:05.637120 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:05.637234 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:05.637809 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:04:05.637820 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:04:05.639673 (+ 1853us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:04:05.641111 (+ 1438us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:04:05.641119 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:04:05.641122 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:04:05.641346 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:04:05.641529 (+ 183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:04:05.641529 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:04:05.641530 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:04:05.641642 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:04:05.641872 (+ 230us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:04:05.641878 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:04:05.642054 (+ 176us) client_negotiation.cc:769] Sending connection context
0812 21:04:05.642148 (+ 94us) client_negotiation.cc:240] Negotiation successful
0812 21:04:05.642309 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":81,"threads_started":1}
I20260812 21:04:05.642993 26197 heartbeater.cc:347] Connected to a master server at 127.11.254.190:43839
I20260812 21:04:05.643078 26197 heartbeater.cc:464] Registering TS with master...
I20260812 21:04:05.643283 26197 heartbeater.cc:511] Master 127.11.254.190:43839 requested a full tablet report, sending...
I20260812 21:04:05.643667 25633 ts_manager.cc:194] Registered new tserver with Master: 868e98cd6025463aa10e6b78dff569a7 (127.11.254.130:35013)
I20260812 21:04:05.643957 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 25998
I20260812 21:04:05.644158 12282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568607354921-12282-0/minicluster-data/ts-1/wal/instance
I20260812 21:04:05.644163 25633 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.11.254.130@KRBTEST.COM'} at 127.11.254.130:49787
I20260812 21:04:05.644483 12282 external_mini_cluster.cc:1428] Setting key 7bf0e1f07379c638dc8adb33a68a12b7
I20260812 21:04:05.650208 25724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:05.644074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.130:56515 (local address 127.11.254.190:43839)
0812 21:04:05.644112 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0812 21:04:05.644113 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 21:04:05.644790 (+ 677us) server_negotiation.cc:407] Connection header received
0812 21:04:05.644851 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:04:05.644852 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:04:05.644863 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:04:05.644916 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:04:05.645268 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:05.645876 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:05.646588 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:05.646852 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:05.646920 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:05.647833 (+ 913us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:04:05.647834 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:04:05.647835 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:04:05.647840 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:04:05.649026 (+ 1186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:04:05.649417 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:04:05.649418 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:04:05.649419 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:04:05.649473 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:04:05.649674 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:04:05.649674 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:04:05.649674 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:04:05.649816 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:04:05.649870 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0812 21:04:05.650066 (+ 196us) server_negotiation.cc:299] Negotiation successful
0812 21:04:05.650141 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 21:04:05.650250 26210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:05.644016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:43839 (local address 127.11.254.130:56515)
0812 21:04:05.644727 (+ 711us) negotiation.cc:107] Waiting for socket to connect
0812 21:04:05.644739 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 21:04:05.644802 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:04:05.644928 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:04:05.644928 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:04:05.645137 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:04:05.645203 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:05.645205 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:05.645981 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:05.645981 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:05.646518 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:05.646520 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:05.646932 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:05.646933 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:05.647022 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:05.647389 (+ 367us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:04:05.647394 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:04:05.647750 (+ 356us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:04:05.649111 (+ 1361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:04:05.649113 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:04:05.649113 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:04:05.649347 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:04:05.649539 (+ 192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:04:05.649540 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:04:05.649540 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:04:05.649610 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:04:05.649875 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:04:05.649876 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:04:05.649931 (+ 55us) client_negotiation.cc:769] Sending connection context
0812 21:04:05.650027 (+ 96us) client_negotiation.cc:240] Negotiation successful
0812 21:04:05.650145 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":684,"thread_start_us":66,"threads_started":1}
WARNING: no policy specified for oryx/127.11.254.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.254.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.254.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568607354921-12282-0/krb5kdc/oryx_127.11.254.131.keytab.
Entry for principal oryx/127.11.254.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568607354921-12282-0/krb5kdc/oryx_127.11.254.131.keytab.
I20260812 21:04:05.673372 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568607354921-12282-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568607354921-12282-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568607354921-12282-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568607354921-12282-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.254.131:0
--local_ip_for_outbound_sockets=127.11.254.131
--webserver_interface=127.11.254.131
--webserver_port=0
--tserver_master_addrs=127.11.254.190:43839
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568607354921-12282-0/krb5kdc/oryx_127.11.254.131.keytab
--principal=oryx/127.11.254.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.254.148:35883
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568607354921-12282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568607354921-12282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568607354921-12282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:04:05.757395 26217 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:04:05.757637 26217 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:04:05.757671 26217 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:04:05.758898 26217 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:04:05.758939 26217 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:04:05.758960 26217 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 21:04:05.758981 26217 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.254.131
I20260812 21:04:05.760484 26217 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.254.148:35883
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568607354921-12282-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568607354921-12282-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568607354921-12282-0/krb5kdc/oryx_127.11.254.131.keytab
--principal=oryx/127.11.254.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.254.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568607354921-12282-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.254.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.11.254.190:43839
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26217
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.254.131
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568607354921-12282-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:04:05.760689 26217 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:04:05.760879 26217 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:04:05.766065 26229 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:04:05.766111 26227 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:04:05.766085 26217 server_base.cc:1034] running on GCE node
W20260812 21:04:05.766091 26231 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:04:05.766510 26217 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:04:05.766746 26217 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:04:05.767902 26217 hybrid_clock.cc:648] HybridClock initialized: now 1786568645767887 us; error 31 us; skew 500 ppm
Aug 12 21:04:05 dist-test-slave-vpvm krb5kdc[25571](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568645, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.254.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:04:05.770538 26217 init.cc:377] Logged in from keytab as oryx/127.11.254.131@KRBTEST.COM (short username oryx)
I20260812 21:04:05.770993 26217 webserver.cc:460] Webserver started at http://127.11.254.131:40349/ using document root <none> and password file <none>
I20260812 21:04:05.771216 26217 fs_manager.cc:359] Metadata directory not provided
I20260812 21:04:05.771291 26217 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:04:05.771420 26217 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:04:05.772586 26217 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568607354921-12282-0/minicluster-data/ts-2/data/instance:
uuid: "6abfe635d04e4e0d85de54a7c7efce63"
format_stamp: "Formatted at 2026-08-12 21:04:05 on dist-test-slave-vpvm"
server_key: "f1f994c255055413aae2ccbba428610d"
server_key_iv: "3fb2c1220e280a1925906a7812fffd3a"
server_key_version: "encryptionkey@0"
I20260812 21:04:05.772928 26217 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568607354921-12282-0/minicluster-data/ts-2/wal/instance:
uuid: "6abfe635d04e4e0d85de54a7c7efce63"
format_stamp: "Formatted at 2026-08-12 21:04:05 on dist-test-slave-vpvm"
server_key: "f1f994c255055413aae2ccbba428610d"
server_key_iv: "3fb2c1220e280a1925906a7812fffd3a"
server_key_version: "encryptionkey@0"
I20260812 21:04:05.774216 26217 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260812 21:04:05.774927 26246 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:04:05.775074 26217 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 21:04:05.775158 26217 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568607354921-12282-0/minicluster-data/ts-2/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568607354921-12282-0/minicluster-data/ts-2/wal
uuid: "6abfe635d04e4e0d85de54a7c7efce63"
format_stamp: "Formatted at 2026-08-12 21:04:05 on dist-test-slave-vpvm"
server_key: "f1f994c255055413aae2ccbba428610d"
server_key_iv: "3fb2c1220e280a1925906a7812fffd3a"
server_key_version: "encryptionkey@0"
I20260812 21:04:05.775225 26217 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568607354921-12282-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568607354921-12282-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568607354921-12282-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:04:05.799424 26217 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:04:05.801339 26217 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:04:05.801498 26217 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:04:05.802069 26217 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:04:05.802114 26217 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:04:05.802145 26217 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:04:05.802168 26217 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 21:04:05 dist-test-slave-vpvm krb5kdc[25571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568645, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.254.131@KRBTEST.COM for oryx/127.11.254.190@KRBTEST.COM
I20260812 21:04:05.809113 26217 rpc_server.cc:307] RPC server started. Bound to: 127.11.254.131:35529
I20260812 21:04:05.809194 26434 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.254.131:35529 every 8 connection(s)
I20260812 21:04:05.809527 26217 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568607354921-12282-0/minicluster-data/ts-2/data/info.pb
I20260812 21:04:05.810992 25724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:05.802511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.131:48827 (local address 127.11.254.190:43839)
0812 21:04:05.802559 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0812 21:04:05.802561 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:04:05.802795 (+ 234us) server_negotiation.cc:407] Connection header received
0812 21:04:05.803297 (+ 502us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:04:05.803299 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:04:05.803313 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:04:05.803384 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:04:05.804036 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:05.804788 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:05.805931 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:05.806161 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:05.806220 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:05.808385 (+ 2165us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:04:05.808386 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:04:05.808386 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:04:05.808395 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:04:05.809940 (+ 1545us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:04:05.810306 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:04:05.810307 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:04:05.810307 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:04:05.810362 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:04:05.810523 (+ 161us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:04:05.810523 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:04:05.810524 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:04:05.810626 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:04:05.810671 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0812 21:04:05.810867 (+ 196us) server_negotiation.cc:299] Negotiation successful
0812 21:04:05.810926 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 21:04:05.811167 26260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:05.802502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:43839 (local address 127.11.254.131:48827)
0812 21:04:05.802668 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0812 21:04:05.802705 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0812 21:04:05.803205 (+ 500us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:04:05.803472 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:04:05.803475 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:04:05.803761 (+ 286us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:04:05.803957 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:05.803963 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:05.804894 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:05.804895 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:05.805839 (+ 944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:05.805842 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:05.806248 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:05.806249 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:05.806375 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:05.806862 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:04:05.806871 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:04:05.808300 (+ 1429us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:04:05.810035 (+ 1735us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:04:05.810039 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:04:05.810041 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:04:05.810247 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:04:05.810399 (+ 152us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:04:05.810400 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:04:05.810400 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:04:05.810483 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:04:05.810661 (+ 178us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:04:05.810663 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:04:05.810774 (+ 111us) client_negotiation.cc:769] Sending connection context
0812 21:04:05.810849 (+ 75us) client_negotiation.cc:240] Negotiation successful
0812 21:04:05.810978 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":76,"threads_started":1}
I20260812 21:04:05.811610 26436 heartbeater.cc:347] Connected to a master server at 127.11.254.190:43839
I20260812 21:04:05.811688 26436 heartbeater.cc:464] Registering TS with master...
I20260812 21:04:05.811868 26436 heartbeater.cc:511] Master 127.11.254.190:43839 requested a full tablet report, sending...
I20260812 21:04:05.812242 25632 ts_manager.cc:194] Registered new tserver with Master: 6abfe635d04e4e0d85de54a7c7efce63 (127.11.254.131:35529)
I20260812 21:04:05.812698 25632 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.11.254.131@KRBTEST.COM'} at 127.11.254.131:48827
I20260812 21:04:05.817656 25724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:05.812668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.131:58847 (local address 127.11.254.190:43839)
0812 21:04:05.812712 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0812 21:04:05.812714 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:04:05.812793 (+ 79us) server_negotiation.cc:407] Connection header received
0812 21:04:05.812867 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:04:05.812868 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:04:05.812878 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:04:05.812924 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:04:05.813294 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:05.813774 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:05.814485 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:05.814670 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:05.814720 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:05.815525 (+ 805us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:04:05.815526 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:04:05.815526 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:04:05.815532 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:04:05.816820 (+ 1288us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:04:05.817139 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:04:05.817140 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:04:05.817140 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:04:05.817186 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:04:05.817316 (+ 130us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:04:05.817317 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:04:05.817317 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:04:05.817404 (+ 87us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:04:05.817446 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0812 21:04:05.817539 (+ 93us) server_negotiation.cc:299] Negotiation successful
0812 21:04:05.817594 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 21:04:05.817672 26442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:05.812605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:43839 (local address 127.11.254.131:58847)
0812 21:04:05.812732 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0812 21:04:05.812743 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 21:04:05.812817 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:04:05.812927 (+ 110us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:04:05.812928 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:04:05.813145 (+ 217us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:04:05.813212 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:05.813214 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:05.813865 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:05.813866 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:05.814420 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:05.814422 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:05.814727 (+ 305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:05.814727 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:05.814775 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:05.815173 (+ 398us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:04:05.815177 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:04:05.815453 (+ 276us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:04:05.816891 (+ 1438us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:04:05.816892 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:04:05.816892 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:04:05.817067 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:04:05.817230 (+ 163us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:04:05.817231 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:04:05.817231 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:04:05.817278 (+ 47us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:04:05.817445 (+ 167us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:04:05.817446 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:04:05.817480 (+ 34us) client_negotiation.cc:769] Sending connection context
0812 21:04:05.817523 (+ 43us) client_negotiation.cc:240] Negotiation successful
0812 21:04:05.817598 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":72,"threads_started":1}
I20260812 21:04:05.819607 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 26217
I20260812 21:04:05.819697 12282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568607354921-12282-0/minicluster-data/ts-2/wal/instance
I20260812 21:04:05.819867 12282 external_mini_cluster.cc:1428] Setting key dbd3bee87f2f7e3980c8e6918e024b27
I20260812 21:04:05.820693 12282 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 12 21:04:05 dist-test-slave-vpvm krb5kdc[25571](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568645, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 21:04:05 dist-test-slave-vpvm krb5kdc[25571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.11.254.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 21:04:05 dist-test-slave-vpvm krb5kdc[25571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.11.254.190@KRBTEST.COM, Server not found in Kerberos database
W20260812 21:04:05.841745 25724 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:04:05.836741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58272 (local address 127.11.254.190:43839)
0812 21:04:05.836791 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0812 21:04:05.836793 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:04:05.836803 (+ 10us) server_negotiation.cc:407] Connection header received
0812 21:04:05.836865 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:04:05.836865 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:04:05.836875 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:04:05.836923 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:04:05.837448 (+ 525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:05.838004 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:05.838706 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:05.838889 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:05.838935 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:05.841657 (+ 2722us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58272: BlockingRecv error: recv got EOF from 127.0.0.1:58272 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
Aug 12 21:04:05 dist-test-slave-vpvm krb5kdc[25571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568645, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.11.254.190@KRBTEST.COM
I20260812 21:04:05.848980 25724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:05.842594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58282 (local address 127.11.254.190:43839)
0812 21:04:05.842670 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0812 21:04:05.842672 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:04:05.842759 (+ 87us) server_negotiation.cc:407] Connection header received
0812 21:04:05.842820 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:04:05.842821 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:04:05.842830 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:04:05.842867 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:04:05.843442 (+ 575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:05.843940 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:05.844677 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:05.844859 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:05.844914 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:05.846934 (+ 2020us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:04:05.846935 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:04:05.846935 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:04:05.846942 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:04:05.848085 (+ 1143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:04:05.848371 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:04:05.848372 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:04:05.848372 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:04:05.848424 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:04:05.848568 (+ 144us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:04:05.848568 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:04:05.848569 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:04:05.848683 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:04:05.848753 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 21:04:05.848850 (+ 97us) server_negotiation.cc:299] Negotiation successful
0812 21:04:05.848910 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260812 21:04:05.850574 25632 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58282:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 21:04:05.852240 25632 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 21:04:05.858928 26474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:05.854831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.131:35529 (local address 127.0.0.1:56292)
0812 21:04:05.855004 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0812 21:04:05.855036 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0812 21:04:05.855099 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:04:05.855418 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:04:05.855420 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:04:05.855423 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:04:05.855492 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:05.855494 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:05.856809 (+ 1315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:05.856810 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:05.857942 (+ 1132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:05.857943 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:05.858603 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:05.858604 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:05.858696 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:05.858711 (+ 15us) client_negotiation.cc:769] Sending connection context
0812 21:04:05.858783 (+ 72us) client_negotiation.cc:240] Negotiation successful
0812 21:04:05.858849 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"mutex_wait_us":41,"thread_start_us":107,"threads_started":1}
I20260812 21:04:05.858938 26477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:05.855077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56292 (local address 127.11.254.131:35529)
0812 21:04:05.855254 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0812 21:04:05.855259 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 21:04:05.855269 (+ 10us) server_negotiation.cc:407] Connection header received
0812 21:04:05.855309 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:04:05.855311 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:04:05.855326 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:04:05.855400 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:04:05.855575 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:05.856718 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:05.858026 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:05.858525 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:05.858609 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:05.858685 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 21:04:05.858801 (+ 116us) server_negotiation.cc:299] Negotiation successful
0812 21:04:05.858844 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":110,"threads_started":1}
I20260812 21:04:05.859071 26478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:05.855211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35804 (local address 127.11.254.129:36765)
0812 21:04:05.855367 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0812 21:04:05.855371 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 21:04:05.855382 (+ 11us) server_negotiation.cc:407] Connection header received
0812 21:04:05.855421 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:04:05.855424 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:04:05.855438 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:04:05.855507 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:04:05.855734 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:05.856803 (+ 1069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:05.857834 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:05.858571 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:05.858853 (+ 282us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:05.858911 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0812 21:04:05.858935 (+ 24us) server_negotiation.cc:299] Negotiation successful
0812 21:04:05.858976 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":78,"threads_started":1}
I20260812 21:04:05.859238 26476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:05.855007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58964 (local address 127.11.254.130:35013)
0812 21:04:05.855304 (+ 297us) server_negotiation.cc:206] Beginning negotiation
0812 21:04:05.855308 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 21:04:05.855487 (+ 179us) server_negotiation.cc:407] Connection header received
0812 21:04:05.855527 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:04:05.855529 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:04:05.855542 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:04:05.855607 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:04:05.855790 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:05.856493 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:05.857986 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:05.858647 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:05.858719 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:05.858774 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0812 21:04:05.859121 (+ 347us) server_negotiation.cc:299] Negotiation successful
0812 21:04:05.859170 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":262,"thread_start_us":82,"threads_started":1}
I20260812 21:04:05.859037 26471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:05.854831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.130:35013 (local address 127.0.0.1:58964)
0812 21:04:05.855366 (+ 535us) negotiation.cc:107] Waiting for socket to connect
0812 21:04:05.855376 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 21:04:05.855431 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:04:05.855668 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:04:05.855669 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:04:05.855671 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:04:05.855717 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:05.855719 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:05.856575 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:05.856575 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:05.857882 (+ 1307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:05.857886 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:05.858806 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:05.858807 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:05.858859 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:05.858863 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 21:04:05.858907 (+ 44us) client_negotiation.cc:240] Negotiation successful
0812 21:04:05.858960 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":524,"mutex_wait_us":35,"thread_start_us":47,"threads_started":1}
I20260812 21:04:05.859448 26473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:05.854829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.129:36765 (local address 127.0.0.1:35804)
0812 21:04:05.854993 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0812 21:04:05.855036 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 21:04:05.855099 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:04:05.855608 (+ 509us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:04:05.855609 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:04:05.855611 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:04:05.855656 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:05.855658 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:05.856874 (+ 1216us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:05.856874 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:05.857766 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:05.857769 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:05.858688 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:05.858689 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:05.858749 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:05.858753 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 21:04:05.858782 (+ 29us) client_negotiation.cc:240] Negotiation successful
0812 21:04:05.859404 (+ 622us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"mutex_wait_us":569,"thread_start_us":117,"threads_started":1}
I20260812 21:04:05.860242 25880 tablet_service.cc:1467] Processing CreateTablet for tablet 4d6aefcfbffa434d8811cb168c248ace (DEFAULT_TABLE table=test-table [id=c2c4431739ff4d92954709b3f8b23b3b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:04:05.860245 26327 tablet_service.cc:1467] Processing CreateTablet for tablet 4d6aefcfbffa434d8811cb168c248ace (DEFAULT_TABLE table=test-table [id=c2c4431739ff4d92954709b3f8b23b3b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:04:05.860244 26097 tablet_service.cc:1467] Processing CreateTablet for tablet 4d6aefcfbffa434d8811cb168c248ace (DEFAULT_TABLE table=test-table [id=c2c4431739ff4d92954709b3f8b23b3b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:04:05.860546 25880 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4d6aefcfbffa434d8811cb168c248ace. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:04:05.860587 26327 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4d6aefcfbffa434d8811cb168c248ace. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:04:05.860587 26097 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4d6aefcfbffa434d8811cb168c248ace. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:04:05.863603 26479 tablet_bootstrap.cc:492] T 4d6aefcfbffa434d8811cb168c248ace P 0d68d6ebf5ac47cfa96cfe32cc083243: Bootstrap starting.
I20260812 21:04:05.864151 26479 tablet_bootstrap.cc:654] T 4d6aefcfbffa434d8811cb168c248ace P 0d68d6ebf5ac47cfa96cfe32cc083243: Neither blocks nor log segments found. Creating new log.
I20260812 21:04:05.864354 26481 tablet_bootstrap.cc:492] T 4d6aefcfbffa434d8811cb168c248ace P 6abfe635d04e4e0d85de54a7c7efce63: Bootstrap starting.
I20260812 21:04:05.864449 26479 log.cc:824] T 4d6aefcfbffa434d8811cb168c248ace P 0d68d6ebf5ac47cfa96cfe32cc083243: Log is configured to *not* fsync() on all Append() calls
I20260812 21:04:05.864751 26484 tablet_bootstrap.cc:492] T 4d6aefcfbffa434d8811cb168c248ace P 868e98cd6025463aa10e6b78dff569a7: Bootstrap starting.
I20260812 21:04:05.864892 26481 tablet_bootstrap.cc:654] T 4d6aefcfbffa434d8811cb168c248ace P 6abfe635d04e4e0d85de54a7c7efce63: Neither blocks nor log segments found. Creating new log.
I20260812 21:04:05.865202 26481 log.cc:824] T 4d6aefcfbffa434d8811cb168c248ace P 6abfe635d04e4e0d85de54a7c7efce63: Log is configured to *not* fsync() on all Append() calls
I20260812 21:04:05.865264 26479 tablet_bootstrap.cc:492] T 4d6aefcfbffa434d8811cb168c248ace P 0d68d6ebf5ac47cfa96cfe32cc083243: No bootstrap required, opened a new log
I20260812 21:04:05.865321 26479 ts_tablet_manager.cc:1397] T 4d6aefcfbffa434d8811cb168c248ace P 0d68d6ebf5ac47cfa96cfe32cc083243: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 21:04:05.865391 26484 tablet_bootstrap.cc:654] T 4d6aefcfbffa434d8811cb168c248ace P 868e98cd6025463aa10e6b78dff569a7: Neither blocks nor log segments found. Creating new log.
I20260812 21:04:05.865722 26484 log.cc:824] T 4d6aefcfbffa434d8811cb168c248ace P 868e98cd6025463aa10e6b78dff569a7: Log is configured to *not* fsync() on all Append() calls
I20260812 21:04:05.866150 26481 tablet_bootstrap.cc:492] T 4d6aefcfbffa434d8811cb168c248ace P 6abfe635d04e4e0d85de54a7c7efce63: No bootstrap required, opened a new log
I20260812 21:04:05.866226 26481 ts_tablet_manager.cc:1397] T 4d6aefcfbffa434d8811cb168c248ace P 6abfe635d04e4e0d85de54a7c7efce63: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 21:04:05.866454 26484 tablet_bootstrap.cc:492] T 4d6aefcfbffa434d8811cb168c248ace P 868e98cd6025463aa10e6b78dff569a7: No bootstrap required, opened a new log
I20260812 21:04:05.866511 26484 ts_tablet_manager.cc:1397] T 4d6aefcfbffa434d8811cb168c248ace P 868e98cd6025463aa10e6b78dff569a7: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 21:04:05.867503 26479 raft_consensus.cc:348] T 4d6aefcfbffa434d8811cb168c248ace P 0d68d6ebf5ac47cfa96cfe32cc083243 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d68d6ebf5ac47cfa96cfe32cc083243" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 36765 } } peers { permanent_uuid: "868e98cd6025463aa10e6b78dff569a7" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 35013 } } peers { permanent_uuid: "6abfe635d04e4e0d85de54a7c7efce63" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 35529 } }
I20260812 21:04:05.867694 26479 raft_consensus.cc:374] T 4d6aefcfbffa434d8811cb168c248ace P 0d68d6ebf5ac47cfa96cfe32cc083243 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:04:05.867729 26479 raft_consensus.cc:729] T 4d6aefcfbffa434d8811cb168c248ace P 0d68d6ebf5ac47cfa96cfe32cc083243 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d68d6ebf5ac47cfa96cfe32cc083243, State: Initialized, Role: FOLLOWER
I20260812 21:04:05.867868 26479 consensus_queue.cc:260] T 4d6aefcfbffa434d8811cb168c248ace P 0d68d6ebf5ac47cfa96cfe32cc083243 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d68d6ebf5ac47cfa96cfe32cc083243" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 36765 } } peers { permanent_uuid: "868e98cd6025463aa10e6b78dff569a7" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 35013 } } peers { permanent_uuid: "6abfe635d04e4e0d85de54a7c7efce63" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 35529 } }
I20260812 21:04:05.867933 26484 raft_consensus.cc:348] T 4d6aefcfbffa434d8811cb168c248ace P 868e98cd6025463aa10e6b78dff569a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d68d6ebf5ac47cfa96cfe32cc083243" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 36765 } } peers { permanent_uuid: "868e98cd6025463aa10e6b78dff569a7" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 35013 } } peers { permanent_uuid: "6abfe635d04e4e0d85de54a7c7efce63" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 35529 } }
I20260812 21:04:05.868093 26484 raft_consensus.cc:374] T 4d6aefcfbffa434d8811cb168c248ace P 868e98cd6025463aa10e6b78dff569a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:04:05.868131 26484 raft_consensus.cc:729] T 4d6aefcfbffa434d8811cb168c248ace P 868e98cd6025463aa10e6b78dff569a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 868e98cd6025463aa10e6b78dff569a7, State: Initialized, Role: FOLLOWER
I20260812 21:04:05.868129 26479 ts_tablet_manager.cc:1428] T 4d6aefcfbffa434d8811cb168c248ace P 0d68d6ebf5ac47cfa96cfe32cc083243: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 21:04:05.868127 26481 raft_consensus.cc:348] T 4d6aefcfbffa434d8811cb168c248ace P 6abfe635d04e4e0d85de54a7c7efce63 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d68d6ebf5ac47cfa96cfe32cc083243" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 36765 } } peers { permanent_uuid: "868e98cd6025463aa10e6b78dff569a7" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 35013 } } peers { permanent_uuid: "6abfe635d04e4e0d85de54a7c7efce63" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 35529 } }
I20260812 21:04:05.868243 25985 heartbeater.cc:503] Master 127.11.254.190:43839 was elected leader, sending a full tablet report...
I20260812 21:04:05.868278 26481 raft_consensus.cc:374] T 4d6aefcfbffa434d8811cb168c248ace P 6abfe635d04e4e0d85de54a7c7efce63 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:04:05.868255 26484 consensus_queue.cc:260] T 4d6aefcfbffa434d8811cb168c248ace P 868e98cd6025463aa10e6b78dff569a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d68d6ebf5ac47cfa96cfe32cc083243" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 36765 } } peers { permanent_uuid: "868e98cd6025463aa10e6b78dff569a7" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 35013 } } peers { permanent_uuid: "6abfe635d04e4e0d85de54a7c7efce63" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 35529 } }
I20260812 21:04:05.868321 26481 raft_consensus.cc:729] T 4d6aefcfbffa434d8811cb168c248ace P 6abfe635d04e4e0d85de54a7c7efce63 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6abfe635d04e4e0d85de54a7c7efce63, State: Initialized, Role: FOLLOWER
I20260812 21:04:05.868424 26481 consensus_queue.cc:260] T 4d6aefcfbffa434d8811cb168c248ace P 6abfe635d04e4e0d85de54a7c7efce63 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d68d6ebf5ac47cfa96cfe32cc083243" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 36765 } } peers { permanent_uuid: "868e98cd6025463aa10e6b78dff569a7" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 35013 } } peers { permanent_uuid: "6abfe635d04e4e0d85de54a7c7efce63" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 35529 } }
I20260812 21:04:05.868499 26484 ts_tablet_manager.cc:1428] T 4d6aefcfbffa434d8811cb168c248ace P 868e98cd6025463aa10e6b78dff569a7: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260812 21:04:05.868604 26197 heartbeater.cc:503] Master 127.11.254.190:43839 was elected leader, sending a full tablet report...
I20260812 21:04:05.868669 26481 ts_tablet_manager.cc:1428] T 4d6aefcfbffa434d8811cb168c248ace P 6abfe635d04e4e0d85de54a7c7efce63: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260812 21:04:05.868784 26436 heartbeater.cc:503] Master 127.11.254.190:43839 was elected leader, sending a full tablet report...
W20260812 21:04:05.889966 26199 tablet.cc:2369] T 4d6aefcfbffa434d8811cb168c248ace P 868e98cd6025463aa10e6b78dff569a7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 21:04:05.931682 26490 raft_consensus.cc:482] T 4d6aefcfbffa434d8811cb168c248ace P 868e98cd6025463aa10e6b78dff569a7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 21:04:05.931808 26490 raft_consensus.cc:504] T 4d6aefcfbffa434d8811cb168c248ace P 868e98cd6025463aa10e6b78dff569a7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d68d6ebf5ac47cfa96cfe32cc083243" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 36765 } } peers { permanent_uuid: "868e98cd6025463aa10e6b78dff569a7" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 35013 } } peers { permanent_uuid: "6abfe635d04e4e0d85de54a7c7efce63" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 35529 } }
I20260812 21:04:05.932217 26490 leader_election.cc:288] T 4d6aefcfbffa434d8811cb168c248ace P 868e98cd6025463aa10e6b78dff569a7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0d68d6ebf5ac47cfa96cfe32cc083243 (127.11.254.129:36765), 6abfe635d04e4e0d85de54a7c7efce63 (127.11.254.131:35529)
I20260812 21:04:05.936025 26477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:05.932520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.130:44127 (local address 127.11.254.131:35529)
0812 21:04:05.932580 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0812 21:04:05.932582 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:04:05.932591 (+ 9us) server_negotiation.cc:407] Connection header received
0812 21:04:05.932620 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:04:05.932621 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:04:05.932628 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:04:05.932681 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:04:05.932850 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:05.933624 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:05.935125 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:05.935679 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:05.935755 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:05.935764 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 21:04:05.935935 (+ 171us) server_negotiation.cc:299] Negotiation successful
0812 21:04:05.935967 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260812 21:04:05.936069 26033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:05.932430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.131:35529 (local address 127.11.254.130:44127)
0812 21:04:05.932489 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0812 21:04:05.932500 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 21:04:05.932545 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:04:05.932702 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:04:05.932703 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:04:05.932704 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:04:05.932779 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:05.932781 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:05.933719 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:05.933720 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:05.935027 (+ 1307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:05.935030 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:05.935758 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:05.935758 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:05.935869 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:05.935873 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 21:04:05.935923 (+ 50us) client_negotiation.cc:240] Negotiation successful
0812 21:04:05.935967 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39,"mutex_wait_us":90}
I20260812 21:04:05.936472 26492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:05.932451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.129:36765 (local address 127.11.254.130:50599)
0812 21:04:05.932633 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0812 21:04:05.932642 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 21:04:05.932698 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:04:05.932809 (+ 111us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:04:05.932810 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:04:05.932812 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:04:05.932875 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:05.932877 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:05.933727 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:05.933727 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:05.935027 (+ 1300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:05.935030 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:05.935758 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:05.935758 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:05.936276 (+ 518us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:05.936281 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 21:04:05.936335 (+ 54us) client_negotiation.cc:240] Negotiation successful
0812 21:04:05.936377 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"mutex_wait_us":9,"thread_start_us":103,"threads_started":1}
I20260812 21:04:05.936496 26478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:05.932520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.130:50599 (local address 127.11.254.129:36765)
0812 21:04:05.932561 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0812 21:04:05.932562 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 21:04:05.932705 (+ 143us) server_negotiation.cc:407] Connection header received
0812 21:04:05.932739 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:04:05.932740 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:04:05.932748 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:04:05.932799 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:04:05.932929 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:05.933661 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:05.935125 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:05.935678 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:05.935755 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:05.935764 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 21:04:05.936367 (+ 603us) server_negotiation.cc:299] Negotiation successful
0812 21:04:05.936448 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 21:04:05.936442 26358 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d6aefcfbffa434d8811cb168c248ace" candidate_uuid: "868e98cd6025463aa10e6b78dff569a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6abfe635d04e4e0d85de54a7c7efce63" is_pre_election: true
I20260812 21:04:05.936609 26358 raft_consensus.cc:2454] T 4d6aefcfbffa434d8811cb168c248ace P 6abfe635d04e4e0d85de54a7c7efce63 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 868e98cd6025463aa10e6b78dff569a7 in term 0.
I20260812 21:04:05.936865 26022 leader_election.cc:302] T 4d6aefcfbffa434d8811cb168c248ace P 868e98cd6025463aa10e6b78dff569a7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6abfe635d04e4e0d85de54a7c7efce63, 868e98cd6025463aa10e6b78dff569a7; no voters:
I20260812 21:04:05.936883 25911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d6aefcfbffa434d8811cb168c248ace" candidate_uuid: "868e98cd6025463aa10e6b78dff569a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d68d6ebf5ac47cfa96cfe32cc083243" is_pre_election: true
I20260812 21:04:05.937033 25911 raft_consensus.cc:2454] T 4d6aefcfbffa434d8811cb168c248ace P 0d68d6ebf5ac47cfa96cfe32cc083243 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 868e98cd6025463aa10e6b78dff569a7 in term 0.
I20260812 21:04:05.937003 26490 raft_consensus.cc:2781] T 4d6aefcfbffa434d8811cb168c248ace P 868e98cd6025463aa10e6b78dff569a7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 21:04:05.937074 26490 raft_consensus.cc:482] T 4d6aefcfbffa434d8811cb168c248ace P 868e98cd6025463aa10e6b78dff569a7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 21:04:05.937099 26490 raft_consensus.cc:3037] T 4d6aefcfbffa434d8811cb168c248ace P 868e98cd6025463aa10e6b78dff569a7 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:04:05.937827 26490 raft_consensus.cc:504] T 4d6aefcfbffa434d8811cb168c248ace P 868e98cd6025463aa10e6b78dff569a7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d68d6ebf5ac47cfa96cfe32cc083243" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 36765 } } peers { permanent_uuid: "868e98cd6025463aa10e6b78dff569a7" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 35013 } } peers { permanent_uuid: "6abfe635d04e4e0d85de54a7c7efce63" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 35529 } }
I20260812 21:04:05.937970 26490 leader_election.cc:288] T 4d6aefcfbffa434d8811cb168c248ace P 868e98cd6025463aa10e6b78dff569a7 [CANDIDATE]: Term 1 election: Requested vote from peers 0d68d6ebf5ac47cfa96cfe32cc083243 (127.11.254.129:36765), 6abfe635d04e4e0d85de54a7c7efce63 (127.11.254.131:35529)
I20260812 21:04:05.938133 25911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d6aefcfbffa434d8811cb168c248ace" candidate_uuid: "868e98cd6025463aa10e6b78dff569a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d68d6ebf5ac47cfa96cfe32cc083243"
I20260812 21:04:05.938211 25911 raft_consensus.cc:3037] T 4d6aefcfbffa434d8811cb168c248ace P 0d68d6ebf5ac47cfa96cfe32cc083243 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:04:05.938241 26358 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d6aefcfbffa434d8811cb168c248ace" candidate_uuid: "868e98cd6025463aa10e6b78dff569a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6abfe635d04e4e0d85de54a7c7efce63"
I20260812 21:04:05.938306 26358 raft_consensus.cc:3037] T 4d6aefcfbffa434d8811cb168c248ace P 6abfe635d04e4e0d85de54a7c7efce63 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:04:05.939077 26358 raft_consensus.cc:2454] T 4d6aefcfbffa434d8811cb168c248ace P 6abfe635d04e4e0d85de54a7c7efce63 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 868e98cd6025463aa10e6b78dff569a7 in term 1.
I20260812 21:04:05.939077 25911 raft_consensus.cc:2454] T 4d6aefcfbffa434d8811cb168c248ace P 0d68d6ebf5ac47cfa96cfe32cc083243 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 868e98cd6025463aa10e6b78dff569a7 in term 1.
I20260812 21:04:05.939268 26021 leader_election.cc:302] T 4d6aefcfbffa434d8811cb168c248ace P 868e98cd6025463aa10e6b78dff569a7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d68d6ebf5ac47cfa96cfe32cc083243, 868e98cd6025463aa10e6b78dff569a7; no voters:
I20260812 21:04:05.939360 26490 raft_consensus.cc:2781] T 4d6aefcfbffa434d8811cb168c248ace P 868e98cd6025463aa10e6b78dff569a7 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:04:05.939518 26490 raft_consensus.cc:686] T 4d6aefcfbffa434d8811cb168c248ace P 868e98cd6025463aa10e6b78dff569a7 [term 1 LEADER]: Becoming Leader. State: Replica: 868e98cd6025463aa10e6b78dff569a7, State: Running, Role: LEADER
I20260812 21:04:05.939599 26490 consensus_queue.cc:237] T 4d6aefcfbffa434d8811cb168c248ace P 868e98cd6025463aa10e6b78dff569a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d68d6ebf5ac47cfa96cfe32cc083243" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 36765 } } peers { permanent_uuid: "868e98cd6025463aa10e6b78dff569a7" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 35013 } } peers { permanent_uuid: "6abfe635d04e4e0d85de54a7c7efce63" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 35529 } }
I20260812 21:04:05.940426 25633 catalog_manager.cc:5495] T 4d6aefcfbffa434d8811cb168c248ace P 868e98cd6025463aa10e6b78dff569a7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 868e98cd6025463aa10e6b78dff569a7 (127.11.254.130). New cstate: current_term: 1 leader_uuid: "868e98cd6025463aa10e6b78dff569a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d68d6ebf5ac47cfa96cfe32cc083243" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 36765 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "868e98cd6025463aa10e6b78dff569a7" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 35013 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6abfe635d04e4e0d85de54a7c7efce63" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 35529 } health_report { overall_health: UNKNOWN } } }
W20260812 21:04:05.947103 25986 tablet.cc:2369] T 4d6aefcfbffa434d8811cb168c248ace P 0d68d6ebf5ac47cfa96cfe32cc083243: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 21:04:05.950529 26476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:05.948313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58978 (local address 127.11.254.130:35013)
0812 21:04:05.948361 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0812 21:04:05.948363 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:04:05.948371 (+ 8us) server_negotiation.cc:407] Connection header received
0812 21:04:05.948401 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:04:05.948402 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:04:05.948419 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:04:05.948471 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 21:04:05.948571 (+ 100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:05.949095 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:05.949837 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:05.950058 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:05.950110 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:05.950252 (+ 142us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 21:04:05.950350 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 21:04:05.950409 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0812 21:04:05.950455 (+ 46us) server_negotiation.cc:299] Negotiation successful
0812 21:04:05.950474 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260812 21:04:05.952263 25911 raft_consensus.cc:1261] T 4d6aefcfbffa434d8811cb168c248ace P 0d68d6ebf5ac47cfa96cfe32cc083243 [term 1 FOLLOWER]: Refusing update from remote peer 868e98cd6025463aa10e6b78dff569a7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 21:04:05.952263 26358 raft_consensus.cc:1261] T 4d6aefcfbffa434d8811cb168c248ace P 6abfe635d04e4e0d85de54a7c7efce63 [term 1 FOLLOWER]: Refusing update from remote peer 868e98cd6025463aa10e6b78dff569a7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 21:04:05.952610 26490 consensus_queue.cc:1035] T 4d6aefcfbffa434d8811cb168c248ace P 868e98cd6025463aa10e6b78dff569a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d68d6ebf5ac47cfa96cfe32cc083243" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 36765 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 21:04:05.952751 26494 consensus_queue.cc:1035] T 4d6aefcfbffa434d8811cb168c248ace P 868e98cd6025463aa10e6b78dff569a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6abfe635d04e4e0d85de54a7c7efce63" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 35529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 21:04:05.955951 26498 mvcc.cc:205] Tried to move back new op lower bound from 7317785173817917440 to 7317785173769207808. Current Snapshot: MvccSnapshot[applied={T|T < 7317785173817917440 or (T in {7317785173817917440})}]
I20260812 21:04:05.957247 25633 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58282:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 21:04:05.957331 25633 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58282:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 21:04:05.958094 25633 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5eb7e77e033b4ec48c7dabe9b94175eb: Sending DeleteTablet for 3 replicas of tablet 4d6aefcfbffa434d8811cb168c248ace
I20260812 21:04:05.958427 25880 tablet_service.cc:1514] Processing DeleteTablet for tablet 4d6aefcfbffa434d8811cb168c248ace with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:04:05 UTC) from {username='oryx', principal='oryx/127.11.254.190@KRBTEST.COM'} at 127.0.0.1:35804
I20260812 21:04:05.958473 26097 tablet_service.cc:1514] Processing DeleteTablet for tablet 4d6aefcfbffa434d8811cb168c248ace with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:04:05 UTC) from {username='oryx', principal='oryx/127.11.254.190@KRBTEST.COM'} at 127.0.0.1:58964
I20260812 21:04:05.958607 26516 tablet_replica.cc:330] T 4d6aefcfbffa434d8811cb168c248ace P 0d68d6ebf5ac47cfa96cfe32cc083243: stopping tablet replica
I20260812 21:04:05.958611 26327 tablet_service.cc:1514] Processing DeleteTablet for tablet 4d6aefcfbffa434d8811cb168c248ace with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:04:05 UTC) from {username='oryx', principal='oryx/127.11.254.190@KRBTEST.COM'} at 127.0.0.1:56292
I20260812 21:04:05.958750 26516 raft_consensus.cc:2229] T 4d6aefcfbffa434d8811cb168c248ace P 0d68d6ebf5ac47cfa96cfe32cc083243 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 21:04:05.958822 26519 tablet_replica.cc:330] T 4d6aefcfbffa434d8811cb168c248ace P 6abfe635d04e4e0d85de54a7c7efce63: stopping tablet replica
I20260812 21:04:05.958926 26519 raft_consensus.cc:2229] T 4d6aefcfbffa434d8811cb168c248ace P 6abfe635d04e4e0d85de54a7c7efce63 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 21:04:05.959036 26519 raft_consensus.cc:2258] T 4d6aefcfbffa434d8811cb168c248ace P 6abfe635d04e4e0d85de54a7c7efce63 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 21:04:05.959044 25633 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58282:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 21:04:05.959208 25633 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 21:04:05.958952 26516 raft_consensus.cc:2258] T 4d6aefcfbffa434d8811cb168c248ace P 0d68d6ebf5ac47cfa96cfe32cc083243 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 21:04:05.959424 26517 tablet_replica.cc:330] T 4d6aefcfbffa434d8811cb168c248ace P 868e98cd6025463aa10e6b78dff569a7: stopping tablet replica
I20260812 21:04:05.959512 26517 raft_consensus.cc:2229] T 4d6aefcfbffa434d8811cb168c248ace P 868e98cd6025463aa10e6b78dff569a7 [term 1 LEADER]: Raft consensus shutting down.
I20260812 21:04:05.959612 26517 raft_consensus.cc:2258] T 4d6aefcfbffa434d8811cb168c248ace P 868e98cd6025463aa10e6b78dff569a7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 21:04:05.959952 26519 ts_tablet_manager.cc:1905] T 4d6aefcfbffa434d8811cb168c248ace P 6abfe635d04e4e0d85de54a7c7efce63: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 21:04:05.960453 26517 ts_tablet_manager.cc:1905] T 4d6aefcfbffa434d8811cb168c248ace P 868e98cd6025463aa10e6b78dff569a7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 21:04:05.960598 25880 tablet_service.cc:1467] Processing CreateTablet for tablet 8ce6a346634047758996dda6db22ac65 (DEFAULT_TABLE table=test-table [id=4427eb72b7d542238e5d5b4e6f4ff490]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:04:05.960727 25880 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8ce6a346634047758996dda6db22ac65. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:04:05.962066 26327 tablet_service.cc:1467] Processing CreateTablet for tablet 8ce6a346634047758996dda6db22ac65 (DEFAULT_TABLE table=test-table [id=4427eb72b7d542238e5d5b4e6f4ff490]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:04:05.962177 26479 tablet_bootstrap.cc:492] T 8ce6a346634047758996dda6db22ac65 P 0d68d6ebf5ac47cfa96cfe32cc083243: Bootstrap starting.
I20260812 21:04:05.962285 26327 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8ce6a346634047758996dda6db22ac65. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:04:05.962245 26517 ts_tablet_manager.cc:1918] T 4d6aefcfbffa434d8811cb168c248ace P 868e98cd6025463aa10e6b78dff569a7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 21:04:05.962488 26517 log.cc:1196] T 4d6aefcfbffa434d8811cb168c248ace P 868e98cd6025463aa10e6b78dff569a7: Deleting WAL directory at /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568607354921-12282-0/minicluster-data/ts-1/wal/wals/4d6aefcfbffa434d8811cb168c248ace
I20260812 21:04:05.962545 26479 tablet_bootstrap.cc:654] T 8ce6a346634047758996dda6db22ac65 P 0d68d6ebf5ac47cfa96cfe32cc083243: Neither blocks nor log segments found. Creating new log.
I20260812 21:04:05.962726 26517 ts_tablet_manager.cc:1939] T 4d6aefcfbffa434d8811cb168c248ace P 868e98cd6025463aa10e6b78dff569a7: Deleting consensus metadata
I20260812 21:04:05.963433 26481 tablet_bootstrap.cc:492] T 8ce6a346634047758996dda6db22ac65 P 6abfe635d04e4e0d85de54a7c7efce63: Bootstrap starting.
I20260812 21:04:05.963593 26479 tablet_bootstrap.cc:492] T 8ce6a346634047758996dda6db22ac65 P 0d68d6ebf5ac47cfa96cfe32cc083243: No bootstrap required, opened a new log
I20260812 21:04:05.963646 26479 ts_tablet_manager.cc:1397] T 8ce6a346634047758996dda6db22ac65 P 0d68d6ebf5ac47cfa96cfe32cc083243: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 21:04:05.963624 25617 catalog_manager.cc:4841] TS 868e98cd6025463aa10e6b78dff569a7 (127.11.254.130:35013): tablet 4d6aefcfbffa434d8811cb168c248ace (table test-table [id=c2c4431739ff4d92954709b3f8b23b3b]) successfully deleted
I20260812 21:04:05.963778 26097 tablet_service.cc:1467] Processing CreateTablet for tablet 8ce6a346634047758996dda6db22ac65 (DEFAULT_TABLE table=test-table [id=4427eb72b7d542238e5d5b4e6f4ff490]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:04:05.963814 26481 tablet_bootstrap.cc:654] T 8ce6a346634047758996dda6db22ac65 P 6abfe635d04e4e0d85de54a7c7efce63: Neither blocks nor log segments found. Creating new log.
I20260812 21:04:05.963795 26479 raft_consensus.cc:348] T 8ce6a346634047758996dda6db22ac65 P 0d68d6ebf5ac47cfa96cfe32cc083243 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d68d6ebf5ac47cfa96cfe32cc083243" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 36765 } } peers { permanent_uuid: "868e98cd6025463aa10e6b78dff569a7" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 35013 } } peers { permanent_uuid: "6abfe635d04e4e0d85de54a7c7efce63" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 35529 } }
I20260812 21:04:05.963863 26479 raft_consensus.cc:374] T 8ce6a346634047758996dda6db22ac65 P 0d68d6ebf5ac47cfa96cfe32cc083243 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:04:05.963876 26097 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8ce6a346634047758996dda6db22ac65. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:04:05.963886 26479 raft_consensus.cc:729] T 8ce6a346634047758996dda6db22ac65 P 0d68d6ebf5ac47cfa96cfe32cc083243 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d68d6ebf5ac47cfa96cfe32cc083243, State: Initialized, Role: FOLLOWER
I20260812 21:04:05.963934 26479 consensus_queue.cc:260] T 8ce6a346634047758996dda6db22ac65 P 0d68d6ebf5ac47cfa96cfe32cc083243 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d68d6ebf5ac47cfa96cfe32cc083243" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 36765 } } peers { permanent_uuid: "868e98cd6025463aa10e6b78dff569a7" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 35013 } } peers { permanent_uuid: "6abfe635d04e4e0d85de54a7c7efce63" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 35529 } }
I20260812 21:04:05.964033 26479 ts_tablet_manager.cc:1428] T 8ce6a346634047758996dda6db22ac65 P 0d68d6ebf5ac47cfa96cfe32cc083243: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260812 21:04:05.964633 26481 tablet_bootstrap.cc:492] T 8ce6a346634047758996dda6db22ac65 P 6abfe635d04e4e0d85de54a7c7efce63: No bootstrap required, opened a new log
I20260812 21:04:05.964677 26481 ts_tablet_manager.cc:1397] T 8ce6a346634047758996dda6db22ac65 P 6abfe635d04e4e0d85de54a7c7efce63: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 21:04:05.964809 26481 raft_consensus.cc:348] T 8ce6a346634047758996dda6db22ac65 P 6abfe635d04e4e0d85de54a7c7efce63 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d68d6ebf5ac47cfa96cfe32cc083243" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 36765 } } peers { permanent_uuid: "868e98cd6025463aa10e6b78dff569a7" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 35013 } } peers { permanent_uuid: "6abfe635d04e4e0d85de54a7c7efce63" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 35529 } }
I20260812 21:04:05.964865 26481 raft_consensus.cc:374] T 8ce6a346634047758996dda6db22ac65 P 6abfe635d04e4e0d85de54a7c7efce63 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:04:05.964882 26481 raft_consensus.cc:729] T 8ce6a346634047758996dda6db22ac65 P 6abfe635d04e4e0d85de54a7c7efce63 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6abfe635d04e4e0d85de54a7c7efce63, State: Initialized, Role: FOLLOWER
I20260812 21:04:05.964922 26481 consensus_queue.cc:260] T 8ce6a346634047758996dda6db22ac65 P 6abfe635d04e4e0d85de54a7c7efce63 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d68d6ebf5ac47cfa96cfe32cc083243" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 36765 } } peers { permanent_uuid: "868e98cd6025463aa10e6b78dff569a7" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 35013 } } peers { permanent_uuid: "6abfe635d04e4e0d85de54a7c7efce63" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 35529 } }
I20260812 21:04:05.965022 26481 ts_tablet_manager.cc:1428] T 8ce6a346634047758996dda6db22ac65 P 6abfe635d04e4e0d85de54a7c7efce63: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260812 21:04:05.965013 26484 tablet_bootstrap.cc:492] T 8ce6a346634047758996dda6db22ac65 P 868e98cd6025463aa10e6b78dff569a7: Bootstrap starting.
I20260812 21:04:05.965389 26519 ts_tablet_manager.cc:1918] T 4d6aefcfbffa434d8811cb168c248ace P 6abfe635d04e4e0d85de54a7c7efce63: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 21:04:05.965428 26484 tablet_bootstrap.cc:654] T 8ce6a346634047758996dda6db22ac65 P 868e98cd6025463aa10e6b78dff569a7: Neither blocks nor log segments found. Creating new log.
I20260812 21:04:05.965462 26519 log.cc:1196] T 4d6aefcfbffa434d8811cb168c248ace P 6abfe635d04e4e0d85de54a7c7efce63: Deleting WAL directory at /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568607354921-12282-0/minicluster-data/ts-2/wal/wals/4d6aefcfbffa434d8811cb168c248ace
I20260812 21:04:05.965723 26519 ts_tablet_manager.cc:1939] T 4d6aefcfbffa434d8811cb168c248ace P 6abfe635d04e4e0d85de54a7c7efce63: Deleting consensus metadata
I20260812 21:04:05.965945 26516 ts_tablet_manager.cc:1905] T 4d6aefcfbffa434d8811cb168c248ace P 0d68d6ebf5ac47cfa96cfe32cc083243: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 21:04:05.966058 26484 tablet_bootstrap.cc:492] T 8ce6a346634047758996dda6db22ac65 P 868e98cd6025463aa10e6b78dff569a7: No bootstrap required, opened a new log
I20260812 21:04:05.966106 26484 ts_tablet_manager.cc:1397] T 8ce6a346634047758996dda6db22ac65 P 868e98cd6025463aa10e6b78dff569a7: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 21:04:05.966212 25615 catalog_manager.cc:4841] TS 6abfe635d04e4e0d85de54a7c7efce63 (127.11.254.131:35529): tablet 4d6aefcfbffa434d8811cb168c248ace (table test-table [id=c2c4431739ff4d92954709b3f8b23b3b]) successfully deleted
I20260812 21:04:05.966249 26484 raft_consensus.cc:348] T 8ce6a346634047758996dda6db22ac65 P 868e98cd6025463aa10e6b78dff569a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d68d6ebf5ac47cfa96cfe32cc083243" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 36765 } } peers { permanent_uuid: "868e98cd6025463aa10e6b78dff569a7" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 35013 } } peers { permanent_uuid: "6abfe635d04e4e0d85de54a7c7efce63" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 35529 } }
I20260812 21:04:05.966320 26484 raft_consensus.cc:374] T 8ce6a346634047758996dda6db22ac65 P 868e98cd6025463aa10e6b78dff569a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:04:05.966343 26484 raft_consensus.cc:729] T 8ce6a346634047758996dda6db22ac65 P 868e98cd6025463aa10e6b78dff569a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 868e98cd6025463aa10e6b78dff569a7, State: Initialized, Role: FOLLOWER
I20260812 21:04:05.966385 26484 consensus_queue.cc:260] T 8ce6a346634047758996dda6db22ac65 P 868e98cd6025463aa10e6b78dff569a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d68d6ebf5ac47cfa96cfe32cc083243" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 36765 } } peers { permanent_uuid: "868e98cd6025463aa10e6b78dff569a7" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 35013 } } peers { permanent_uuid: "6abfe635d04e4e0d85de54a7c7efce63" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 35529 } }
I20260812 21:04:05.966473 26484 ts_tablet_manager.cc:1428] T 8ce6a346634047758996dda6db22ac65 P 868e98cd6025463aa10e6b78dff569a7: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 21:04:05.967319 26516 ts_tablet_manager.cc:1918] T 4d6aefcfbffa434d8811cb168c248ace P 0d68d6ebf5ac47cfa96cfe32cc083243: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 21:04:05.967386 26516 log.cc:1196] T 4d6aefcfbffa434d8811cb168c248ace P 0d68d6ebf5ac47cfa96cfe32cc083243: Deleting WAL directory at /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568607354921-12282-0/minicluster-data/ts-0/wal/wals/4d6aefcfbffa434d8811cb168c248ace
I20260812 21:04:05.967903 26516 ts_tablet_manager.cc:1939] T 4d6aefcfbffa434d8811cb168c248ace P 0d68d6ebf5ac47cfa96cfe32cc083243: Deleting consensus metadata
I20260812 21:04:05.968283 25613 catalog_manager.cc:4841] TS 0d68d6ebf5ac47cfa96cfe32cc083243 (127.11.254.129:36765): tablet 4d6aefcfbffa434d8811cb168c248ace (table test-table [id=c2c4431739ff4d92954709b3f8b23b3b]) successfully deleted
W20260812 21:04:06.060115 26437 tablet.cc:2369] T 8ce6a346634047758996dda6db22ac65 P 6abfe635d04e4e0d85de54a7c7efce63: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 21:04:06.180104 26490 raft_consensus.cc:482] T 8ce6a346634047758996dda6db22ac65 P 868e98cd6025463aa10e6b78dff569a7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 21:04:06.180222 26490 raft_consensus.cc:504] T 8ce6a346634047758996dda6db22ac65 P 868e98cd6025463aa10e6b78dff569a7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d68d6ebf5ac47cfa96cfe32cc083243" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 36765 } } peers { permanent_uuid: "868e98cd6025463aa10e6b78dff569a7" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 35013 } } peers { permanent_uuid: "6abfe635d04e4e0d85de54a7c7efce63" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 35529 } }
I20260812 21:04:06.180421 26490 leader_election.cc:288] T 8ce6a346634047758996dda6db22ac65 P 868e98cd6025463aa10e6b78dff569a7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0d68d6ebf5ac47cfa96cfe32cc083243 (127.11.254.129:36765), 6abfe635d04e4e0d85de54a7c7efce63 (127.11.254.131:35529)
I20260812 21:04:06.180652 26358 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ce6a346634047758996dda6db22ac65" candidate_uuid: "868e98cd6025463aa10e6b78dff569a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6abfe635d04e4e0d85de54a7c7efce63" is_pre_election: true
I20260812 21:04:06.180671 25911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ce6a346634047758996dda6db22ac65" candidate_uuid: "868e98cd6025463aa10e6b78dff569a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d68d6ebf5ac47cfa96cfe32cc083243" is_pre_election: true
I20260812 21:04:06.180744 26358 raft_consensus.cc:2454] T 8ce6a346634047758996dda6db22ac65 P 6abfe635d04e4e0d85de54a7c7efce63 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 868e98cd6025463aa10e6b78dff569a7 in term 0.
I20260812 21:04:06.180783 25911 raft_consensus.cc:2454] T 8ce6a346634047758996dda6db22ac65 P 0d68d6ebf5ac47cfa96cfe32cc083243 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 868e98cd6025463aa10e6b78dff569a7 in term 0.
I20260812 21:04:06.180950 26021 leader_election.cc:302] T 8ce6a346634047758996dda6db22ac65 P 868e98cd6025463aa10e6b78dff569a7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d68d6ebf5ac47cfa96cfe32cc083243, 868e98cd6025463aa10e6b78dff569a7; no voters:
I20260812 21:04:06.181046 26490 raft_consensus.cc:2781] T 8ce6a346634047758996dda6db22ac65 P 868e98cd6025463aa10e6b78dff569a7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 21:04:06.181082 26490 raft_consensus.cc:482] T 8ce6a346634047758996dda6db22ac65 P 868e98cd6025463aa10e6b78dff569a7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 21:04:06.181100 26490 raft_consensus.cc:3037] T 8ce6a346634047758996dda6db22ac65 P 868e98cd6025463aa10e6b78dff569a7 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:04:06.181813 26490 raft_consensus.cc:504] T 8ce6a346634047758996dda6db22ac65 P 868e98cd6025463aa10e6b78dff569a7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d68d6ebf5ac47cfa96cfe32cc083243" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 36765 } } peers { permanent_uuid: "868e98cd6025463aa10e6b78dff569a7" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 35013 } } peers { permanent_uuid: "6abfe635d04e4e0d85de54a7c7efce63" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 35529 } }
I20260812 21:04:06.181943 26490 leader_election.cc:288] T 8ce6a346634047758996dda6db22ac65 P 868e98cd6025463aa10e6b78dff569a7 [CANDIDATE]: Term 1 election: Requested vote from peers 0d68d6ebf5ac47cfa96cfe32cc083243 (127.11.254.129:36765), 6abfe635d04e4e0d85de54a7c7efce63 (127.11.254.131:35529)
I20260812 21:04:06.182106 26358 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ce6a346634047758996dda6db22ac65" candidate_uuid: "868e98cd6025463aa10e6b78dff569a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6abfe635d04e4e0d85de54a7c7efce63"
I20260812 21:04:06.182121 25911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ce6a346634047758996dda6db22ac65" candidate_uuid: "868e98cd6025463aa10e6b78dff569a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d68d6ebf5ac47cfa96cfe32cc083243"
I20260812 21:04:06.182183 26358 raft_consensus.cc:3037] T 8ce6a346634047758996dda6db22ac65 P 6abfe635d04e4e0d85de54a7c7efce63 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:04:06.182190 25911 raft_consensus.cc:3037] T 8ce6a346634047758996dda6db22ac65 P 0d68d6ebf5ac47cfa96cfe32cc083243 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:04:06.182763 25911 raft_consensus.cc:2454] T 8ce6a346634047758996dda6db22ac65 P 0d68d6ebf5ac47cfa96cfe32cc083243 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 868e98cd6025463aa10e6b78dff569a7 in term 1.
I20260812 21:04:06.182888 26358 raft_consensus.cc:2454] T 8ce6a346634047758996dda6db22ac65 P 6abfe635d04e4e0d85de54a7c7efce63 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 868e98cd6025463aa10e6b78dff569a7 in term 1.
I20260812 21:04:06.182935 26021 leader_election.cc:302] T 8ce6a346634047758996dda6db22ac65 P 868e98cd6025463aa10e6b78dff569a7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d68d6ebf5ac47cfa96cfe32cc083243, 868e98cd6025463aa10e6b78dff569a7; no voters:
I20260812 21:04:06.183029 26490 raft_consensus.cc:2781] T 8ce6a346634047758996dda6db22ac65 P 868e98cd6025463aa10e6b78dff569a7 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:04:06.183092 26490 raft_consensus.cc:686] T 8ce6a346634047758996dda6db22ac65 P 868e98cd6025463aa10e6b78dff569a7 [term 1 LEADER]: Becoming Leader. State: Replica: 868e98cd6025463aa10e6b78dff569a7, State: Running, Role: LEADER
I20260812 21:04:06.183162 26490 consensus_queue.cc:237] T 8ce6a346634047758996dda6db22ac65 P 868e98cd6025463aa10e6b78dff569a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d68d6ebf5ac47cfa96cfe32cc083243" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 36765 } } peers { permanent_uuid: "868e98cd6025463aa10e6b78dff569a7" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 35013 } } peers { permanent_uuid: "6abfe635d04e4e0d85de54a7c7efce63" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 35529 } }
I20260812 21:04:06.183744 25632 catalog_manager.cc:5495] T 8ce6a346634047758996dda6db22ac65 P 868e98cd6025463aa10e6b78dff569a7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 868e98cd6025463aa10e6b78dff569a7 (127.11.254.130). New cstate: current_term: 1 leader_uuid: "868e98cd6025463aa10e6b78dff569a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d68d6ebf5ac47cfa96cfe32cc083243" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 36765 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "868e98cd6025463aa10e6b78dff569a7" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 35013 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6abfe635d04e4e0d85de54a7c7efce63" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 35529 } health_report { overall_health: UNKNOWN } } }
Aug 12 21:04:06 dist-test-slave-vpvm krb5kdc[25571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568645, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.254.190@KRBTEST.COM for oryx/127.11.254.190@KRBTEST.COM
I20260812 21:04:06.213807 26536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:06.206741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:43839 (local address 127.0.0.1:58298)
0812 21:04:06.206945 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0812 21:04:06.206957 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 21:04:06.207032 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:04:06.207162 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:04:06.207163 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:04:06.207502 (+ 339us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:04:06.207562 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:06.207565 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:06.208236 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:06.208237 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:06.208774 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:06.208776 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:06.209099 (+ 323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:06.209100 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:06.209148 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:06.209678 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:04:06.209683 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:04:06.211212 (+ 1529us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:04:06.212679 (+ 1467us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:04:06.212687 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:04:06.212690 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:04:06.212872 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:04:06.213044 (+ 172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:04:06.213045 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:04:06.213045 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:04:06.213099 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:04:06.213403 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:04:06.213412 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:04:06.213467 (+ 55us) client_negotiation.cc:769] Sending connection context
0812 21:04:06.213528 (+ 61us) client_negotiation.cc:240] Negotiation successful
0812 21:04:06.213687 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"mutex_wait_us":3,"thread_start_us":112,"threads_started":1}
I20260812 21:04:06.213860 25724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:06.206845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58298 (local address 127.11.254.190:43839)
0812 21:04:06.206932 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0812 21:04:06.206934 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:04:06.207023 (+ 89us) server_negotiation.cc:407] Connection header received
0812 21:04:06.207091 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:04:06.207092 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:04:06.207102 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:04:06.207158 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:04:06.207655 (+ 497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:06.208160 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:06.208855 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:06.209037 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:06.209092 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:06.211304 (+ 2212us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:04:06.211306 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:04:06.211306 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:04:06.211314 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:04:06.212594 (+ 1280us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:04:06.212938 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:04:06.212939 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:04:06.212940 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:04:06.212991 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:04:06.213168 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:04:06.213168 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:04:06.213169 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:04:06.213299 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:04:06.213428 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0812 21:04:06.213558 (+ 130us) server_negotiation.cc:299] Negotiation successful
0812 21:04:06.213757 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":117,"server-negotiator.queue_time_us":61}
I20260812 21:04:06.215819 25632 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.11.254.190@KRBTEST.COM'} at 127.0.0.1:58298:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260812 21:04:06.216012 25632 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 21:04:06.217408 26327 tablet_service.cc:1467] Processing CreateTablet for tablet e6c0e1f4e69c4b7ba464c3442238084f (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=7af00ffe05dd473db5adbabe40f20d13]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 21:04:06.217422 25880 tablet_service.cc:1467] Processing CreateTablet for tablet e6c0e1f4e69c4b7ba464c3442238084f (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=7af00ffe05dd473db5adbabe40f20d13]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 21:04:06.217553 25880 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e6c0e1f4e69c4b7ba464c3442238084f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:04:06.217561 26327 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e6c0e1f4e69c4b7ba464c3442238084f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:04:06.218824 26481 tablet_bootstrap.cc:492] T e6c0e1f4e69c4b7ba464c3442238084f P 6abfe635d04e4e0d85de54a7c7efce63: Bootstrap starting.
I20260812 21:04:06.219094 26479 tablet_bootstrap.cc:492] T e6c0e1f4e69c4b7ba464c3442238084f P 0d68d6ebf5ac47cfa96cfe32cc083243: Bootstrap starting.
I20260812 21:04:06.219208 26097 tablet_service.cc:1467] Processing CreateTablet for tablet e6c0e1f4e69c4b7ba464c3442238084f (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=7af00ffe05dd473db5adbabe40f20d13]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 21:04:06.219293 26481 tablet_bootstrap.cc:654] T e6c0e1f4e69c4b7ba464c3442238084f P 6abfe635d04e4e0d85de54a7c7efce63: Neither blocks nor log segments found. Creating new log.
I20260812 21:04:06.219357 26097 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e6c0e1f4e69c4b7ba464c3442238084f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:04:06.219507 26479 tablet_bootstrap.cc:654] T e6c0e1f4e69c4b7ba464c3442238084f P 0d68d6ebf5ac47cfa96cfe32cc083243: Neither blocks nor log segments found. Creating new log.
I20260812 21:04:06.220072 26479 tablet_bootstrap.cc:492] T e6c0e1f4e69c4b7ba464c3442238084f P 0d68d6ebf5ac47cfa96cfe32cc083243: No bootstrap required, opened a new log
I20260812 21:04:06.220113 26479 ts_tablet_manager.cc:1397] T e6c0e1f4e69c4b7ba464c3442238084f P 0d68d6ebf5ac47cfa96cfe32cc083243: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 21:04:06.220238 26479 raft_consensus.cc:348] T e6c0e1f4e69c4b7ba464c3442238084f P 0d68d6ebf5ac47cfa96cfe32cc083243 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d68d6ebf5ac47cfa96cfe32cc083243" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 36765 } } peers { permanent_uuid: "868e98cd6025463aa10e6b78dff569a7" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 35013 } } peers { permanent_uuid: "6abfe635d04e4e0d85de54a7c7efce63" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 35529 } }
I20260812 21:04:06.220299 26479 raft_consensus.cc:374] T e6c0e1f4e69c4b7ba464c3442238084f P 0d68d6ebf5ac47cfa96cfe32cc083243 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:04:06.220316 26479 raft_consensus.cc:729] T e6c0e1f4e69c4b7ba464c3442238084f P 0d68d6ebf5ac47cfa96cfe32cc083243 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d68d6ebf5ac47cfa96cfe32cc083243, State: Initialized, Role: FOLLOWER
I20260812 21:04:06.220359 26479 consensus_queue.cc:260] T e6c0e1f4e69c4b7ba464c3442238084f P 0d68d6ebf5ac47cfa96cfe32cc083243 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d68d6ebf5ac47cfa96cfe32cc083243" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 36765 } } peers { permanent_uuid: "868e98cd6025463aa10e6b78dff569a7" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 35013 } } peers { permanent_uuid: "6abfe635d04e4e0d85de54a7c7efce63" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 35529 } }
I20260812 21:04:06.220455 26479 ts_tablet_manager.cc:1428] T e6c0e1f4e69c4b7ba464c3442238084f P 0d68d6ebf5ac47cfa96cfe32cc083243: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 21:04:06.220505 26484 tablet_bootstrap.cc:492] T e6c0e1f4e69c4b7ba464c3442238084f P 868e98cd6025463aa10e6b78dff569a7: Bootstrap starting.
I20260812 21:04:06.220542 26489 tablet_replica.cc:439] T e6c0e1f4e69c4b7ba464c3442238084f P 0d68d6ebf5ac47cfa96cfe32cc083243: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d68d6ebf5ac47cfa96cfe32cc083243" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 36765 } } peers { permanent_uuid: "868e98cd6025463aa10e6b78dff569a7" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 35013 } } peers { permanent_uuid: "6abfe635d04e4e0d85de54a7c7efce63" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 35529 } } }
I20260812 21:04:06.220640 26489 tablet_replica.cc:442] T e6c0e1f4e69c4b7ba464c3442238084f P 0d68d6ebf5ac47cfa96cfe32cc083243: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 21:04:06.220805 26484 tablet_bootstrap.cc:654] T e6c0e1f4e69c4b7ba464c3442238084f P 868e98cd6025463aa10e6b78dff569a7: Neither blocks nor log segments found. Creating new log.
I20260812 21:04:06.220840 26481 tablet_bootstrap.cc:492] T e6c0e1f4e69c4b7ba464c3442238084f P 6abfe635d04e4e0d85de54a7c7efce63: No bootstrap required, opened a new log
I20260812 21:04:06.220882 26481 ts_tablet_manager.cc:1397] T e6c0e1f4e69c4b7ba464c3442238084f P 6abfe635d04e4e0d85de54a7c7efce63: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 21:04:06.221022 26481 raft_consensus.cc:348] T e6c0e1f4e69c4b7ba464c3442238084f P 6abfe635d04e4e0d85de54a7c7efce63 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d68d6ebf5ac47cfa96cfe32cc083243" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 36765 } } peers { permanent_uuid: "868e98cd6025463aa10e6b78dff569a7" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 35013 } } peers { permanent_uuid: "6abfe635d04e4e0d85de54a7c7efce63" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 35529 } }
I20260812 21:04:06.221086 26481 raft_consensus.cc:374] T e6c0e1f4e69c4b7ba464c3442238084f P 6abfe635d04e4e0d85de54a7c7efce63 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:04:06.221103 26481 raft_consensus.cc:729] T e6c0e1f4e69c4b7ba464c3442238084f P 6abfe635d04e4e0d85de54a7c7efce63 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6abfe635d04e4e0d85de54a7c7efce63, State: Initialized, Role: FOLLOWER
I20260812 21:04:06.221146 26481 consensus_queue.cc:260] T e6c0e1f4e69c4b7ba464c3442238084f P 6abfe635d04e4e0d85de54a7c7efce63 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d68d6ebf5ac47cfa96cfe32cc083243" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 36765 } } peers { permanent_uuid: "868e98cd6025463aa10e6b78dff569a7" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 35013 } } peers { permanent_uuid: "6abfe635d04e4e0d85de54a7c7efce63" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 35529 } }
I20260812 21:04:06.221239 26481 ts_tablet_manager.cc:1428] T e6c0e1f4e69c4b7ba464c3442238084f P 6abfe635d04e4e0d85de54a7c7efce63: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 21:04:06.221307 26491 tablet_replica.cc:439] T e6c0e1f4e69c4b7ba464c3442238084f P 6abfe635d04e4e0d85de54a7c7efce63: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d68d6ebf5ac47cfa96cfe32cc083243" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 36765 } } peers { permanent_uuid: "868e98cd6025463aa10e6b78dff569a7" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 35013 } } peers { permanent_uuid: "6abfe635d04e4e0d85de54a7c7efce63" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 35529 } } }
I20260812 21:04:06.221447 26491 tablet_replica.cc:442] T e6c0e1f4e69c4b7ba464c3442238084f P 6abfe635d04e4e0d85de54a7c7efce63: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 21:04:06.221629 26484 tablet_bootstrap.cc:492] T e6c0e1f4e69c4b7ba464c3442238084f P 868e98cd6025463aa10e6b78dff569a7: No bootstrap required, opened a new log
I20260812 21:04:06.221681 26484 ts_tablet_manager.cc:1397] T e6c0e1f4e69c4b7ba464c3442238084f P 868e98cd6025463aa10e6b78dff569a7: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260812 21:04:06.221781 26484 raft_consensus.cc:348] T e6c0e1f4e69c4b7ba464c3442238084f P 868e98cd6025463aa10e6b78dff569a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d68d6ebf5ac47cfa96cfe32cc083243" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 36765 } } peers { permanent_uuid: "868e98cd6025463aa10e6b78dff569a7" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 35013 } } peers { permanent_uuid: "6abfe635d04e4e0d85de54a7c7efce63" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 35529 } }
I20260812 21:04:06.221839 26484 raft_consensus.cc:374] T e6c0e1f4e69c4b7ba464c3442238084f P 868e98cd6025463aa10e6b78dff569a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:04:06.221858 26484 raft_consensus.cc:729] T e6c0e1f4e69c4b7ba464c3442238084f P 868e98cd6025463aa10e6b78dff569a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 868e98cd6025463aa10e6b78dff569a7, State: Initialized, Role: FOLLOWER
I20260812 21:04:06.221894 26484 consensus_queue.cc:260] T e6c0e1f4e69c4b7ba464c3442238084f P 868e98cd6025463aa10e6b78dff569a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d68d6ebf5ac47cfa96cfe32cc083243" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 36765 } } peers { permanent_uuid: "868e98cd6025463aa10e6b78dff569a7" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 35013 } } peers { permanent_uuid: "6abfe635d04e4e0d85de54a7c7efce63" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 35529 } }
I20260812 21:04:06.222008 26484 ts_tablet_manager.cc:1428] T e6c0e1f4e69c4b7ba464c3442238084f P 868e98cd6025463aa10e6b78dff569a7: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260812 21:04:06.222013 26490 tablet_replica.cc:439] T e6c0e1f4e69c4b7ba464c3442238084f P 868e98cd6025463aa10e6b78dff569a7: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d68d6ebf5ac47cfa96cfe32cc083243" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 36765 } } peers { permanent_uuid: "868e98cd6025463aa10e6b78dff569a7" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 35013 } } peers { permanent_uuid: "6abfe635d04e4e0d85de54a7c7efce63" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 35529 } } }
I20260812 21:04:06.222152 26490 tablet_replica.cc:442] T e6c0e1f4e69c4b7ba464c3442238084f P 868e98cd6025463aa10e6b78dff569a7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 21:04:06.293517 26489 raft_consensus.cc:482] T e6c0e1f4e69c4b7ba464c3442238084f P 0d68d6ebf5ac47cfa96cfe32cc083243 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 21:04:06.293619 26489 raft_consensus.cc:504] T e6c0e1f4e69c4b7ba464c3442238084f P 0d68d6ebf5ac47cfa96cfe32cc083243 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d68d6ebf5ac47cfa96cfe32cc083243" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 36765 } } peers { permanent_uuid: "868e98cd6025463aa10e6b78dff569a7" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 35013 } } peers { permanent_uuid: "6abfe635d04e4e0d85de54a7c7efce63" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 35529 } }
I20260812 21:04:06.294006 26489 leader_election.cc:288] T e6c0e1f4e69c4b7ba464c3442238084f P 0d68d6ebf5ac47cfa96cfe32cc083243 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 868e98cd6025463aa10e6b78dff569a7 (127.11.254.130:35013), 6abfe635d04e4e0d85de54a7c7efce63 (127.11.254.131:35529)
I20260812 21:04:06.296882 26476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:06.294262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.129:49309 (local address 127.11.254.130:35013)
0812 21:04:06.294309 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0812 21:04:06.294312 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 21:04:06.294424 (+ 112us) server_negotiation.cc:407] Connection header received
0812 21:04:06.294477 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:04:06.294477 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:04:06.294488 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:04:06.294541 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:04:06.294716 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:06.295190 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:06.296214 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:06.296564 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:06.296629 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:06.296637 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 21:04:06.296776 (+ 139us) server_negotiation.cc:299] Negotiation successful
0812 21:04:06.296813 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 21:04:06.296882 26477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:06.294223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.129:36629 (local address 127.11.254.131:35529)
0812 21:04:06.294284 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0812 21:04:06.294286 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:04:06.294411 (+ 125us) server_negotiation.cc:407] Connection header received
0812 21:04:06.294467 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:04:06.294468 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:04:06.294477 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:04:06.294529 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:04:06.294693 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:06.295181 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:06.296211 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:06.296556 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:06.296615 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:06.296624 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 21:04:06.296776 (+ 152us) server_negotiation.cc:299] Negotiation successful
0812 21:04:06.296812 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260812 21:04:06.296904 26541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:06.294179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.130:35013 (local address 127.11.254.129:49309)
0812 21:04:06.294349 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0812 21:04:06.294358 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 21:04:06.294422 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:04:06.294544 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:04:06.294544 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:04:06.294546 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:04:06.294649 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:06.294651 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:06.295268 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:06.295269 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:06.296123 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:06.296125 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:06.296648 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:06.296649 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:06.296710 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:06.296713 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 21:04:06.296758 (+ 45us) client_negotiation.cc:240] Negotiation successful
0812 21:04:06.296802 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"mutex_wait_us":26,"thread_start_us":110,"threads_started":1}
I20260812 21:04:06.296911 26542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:06.294164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.131:35529 (local address 127.11.254.129:36629)
0812 21:04:06.294342 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0812 21:04:06.294352 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 21:04:06.294408 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:04:06.294544 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:04:06.294544 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:04:06.294550 (+ 6us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:04:06.294633 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:06.294635 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:06.295260 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:06.295261 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:06.296119 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:06.296122 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:06.296638 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:06.296639 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:06.296708 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:06.296712 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 21:04:06.296761 (+ 49us) client_negotiation.cc:240] Negotiation successful
0812 21:04:06.296807 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"mutex_wait_us":33,"thread_start_us":90,"threads_started":1}
I20260812 21:04:06.297335 26358 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6c0e1f4e69c4b7ba464c3442238084f" candidate_uuid: "0d68d6ebf5ac47cfa96cfe32cc083243" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6abfe635d04e4e0d85de54a7c7efce63" is_pre_election: true
I20260812 21:04:06.297338 26130 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6c0e1f4e69c4b7ba464c3442238084f" candidate_uuid: "0d68d6ebf5ac47cfa96cfe32cc083243" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "868e98cd6025463aa10e6b78dff569a7" is_pre_election: true
I20260812 21:04:06.297427 26358 raft_consensus.cc:2454] T e6c0e1f4e69c4b7ba464c3442238084f P 6abfe635d04e4e0d85de54a7c7efce63 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d68d6ebf5ac47cfa96cfe32cc083243 in term 0.
I20260812 21:04:06.297461 26130 raft_consensus.cc:2454] T e6c0e1f4e69c4b7ba464c3442238084f P 868e98cd6025463aa10e6b78dff569a7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d68d6ebf5ac47cfa96cfe32cc083243 in term 0.
I20260812 21:04:06.297590 25793 leader_election.cc:302] T e6c0e1f4e69c4b7ba464c3442238084f P 0d68d6ebf5ac47cfa96cfe32cc083243 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d68d6ebf5ac47cfa96cfe32cc083243, 6abfe635d04e4e0d85de54a7c7efce63; no voters:
I20260812 21:04:06.297736 26489 raft_consensus.cc:2781] T e6c0e1f4e69c4b7ba464c3442238084f P 0d68d6ebf5ac47cfa96cfe32cc083243 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 21:04:06.297778 26489 raft_consensus.cc:482] T e6c0e1f4e69c4b7ba464c3442238084f P 0d68d6ebf5ac47cfa96cfe32cc083243 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 21:04:06.297793 26489 raft_consensus.cc:3037] T e6c0e1f4e69c4b7ba464c3442238084f P 0d68d6ebf5ac47cfa96cfe32cc083243 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:04:06.298553 26489 raft_consensus.cc:504] T e6c0e1f4e69c4b7ba464c3442238084f P 0d68d6ebf5ac47cfa96cfe32cc083243 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d68d6ebf5ac47cfa96cfe32cc083243" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 36765 } } peers { permanent_uuid: "868e98cd6025463aa10e6b78dff569a7" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 35013 } } peers { permanent_uuid: "6abfe635d04e4e0d85de54a7c7efce63" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 35529 } }
I20260812 21:04:06.298676 26489 leader_election.cc:288] T e6c0e1f4e69c4b7ba464c3442238084f P 0d68d6ebf5ac47cfa96cfe32cc083243 [CANDIDATE]: Term 1 election: Requested vote from peers 868e98cd6025463aa10e6b78dff569a7 (127.11.254.130:35013), 6abfe635d04e4e0d85de54a7c7efce63 (127.11.254.131:35529)
I20260812 21:04:06.298813 26130 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6c0e1f4e69c4b7ba464c3442238084f" candidate_uuid: "0d68d6ebf5ac47cfa96cfe32cc083243" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "868e98cd6025463aa10e6b78dff569a7"
I20260812 21:04:06.298856 26358 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6c0e1f4e69c4b7ba464c3442238084f" candidate_uuid: "0d68d6ebf5ac47cfa96cfe32cc083243" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6abfe635d04e4e0d85de54a7c7efce63"
I20260812 21:04:06.298894 26130 raft_consensus.cc:3037] T e6c0e1f4e69c4b7ba464c3442238084f P 868e98cd6025463aa10e6b78dff569a7 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:04:06.298918 26358 raft_consensus.cc:3037] T e6c0e1f4e69c4b7ba464c3442238084f P 6abfe635d04e4e0d85de54a7c7efce63 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:04:06.299630 26358 raft_consensus.cc:2454] T e6c0e1f4e69c4b7ba464c3442238084f P 6abfe635d04e4e0d85de54a7c7efce63 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d68d6ebf5ac47cfa96cfe32cc083243 in term 1.
I20260812 21:04:06.299660 26130 raft_consensus.cc:2454] T e6c0e1f4e69c4b7ba464c3442238084f P 868e98cd6025463aa10e6b78dff569a7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d68d6ebf5ac47cfa96cfe32cc083243 in term 1.
I20260812 21:04:06.299800 25797 leader_election.cc:302] T e6c0e1f4e69c4b7ba464c3442238084f P 0d68d6ebf5ac47cfa96cfe32cc083243 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d68d6ebf5ac47cfa96cfe32cc083243, 868e98cd6025463aa10e6b78dff569a7; no voters:
I20260812 21:04:06.299901 26489 raft_consensus.cc:2781] T e6c0e1f4e69c4b7ba464c3442238084f P 0d68d6ebf5ac47cfa96cfe32cc083243 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:04:06.300042 26489 raft_consensus.cc:686] T e6c0e1f4e69c4b7ba464c3442238084f P 0d68d6ebf5ac47cfa96cfe32cc083243 [term 1 LEADER]: Becoming Leader. State: Replica: 0d68d6ebf5ac47cfa96cfe32cc083243, State: Running, Role: LEADER
I20260812 21:04:06.300096 26489 consensus_queue.cc:237] T e6c0e1f4e69c4b7ba464c3442238084f P 0d68d6ebf5ac47cfa96cfe32cc083243 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d68d6ebf5ac47cfa96cfe32cc083243" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 36765 } } peers { permanent_uuid: "868e98cd6025463aa10e6b78dff569a7" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 35013 } } peers { permanent_uuid: "6abfe635d04e4e0d85de54a7c7efce63" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 35529 } }
I20260812 21:04:06.300424 26544 tablet_replica.cc:439] T e6c0e1f4e69c4b7ba464c3442238084f P 0d68d6ebf5ac47cfa96cfe32cc083243: TxnStatusTablet state changed. Reason: New leader 0d68d6ebf5ac47cfa96cfe32cc083243. Latest consensus state: current_term: 1 leader_uuid: "0d68d6ebf5ac47cfa96cfe32cc083243" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d68d6ebf5ac47cfa96cfe32cc083243" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 36765 } } peers { permanent_uuid: "868e98cd6025463aa10e6b78dff569a7" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 35013 } } peers { permanent_uuid: "6abfe635d04e4e0d85de54a7c7efce63" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 35529 } } }
I20260812 21:04:06.300529 26544 tablet_replica.cc:442] T e6c0e1f4e69c4b7ba464c3442238084f P 0d68d6ebf5ac47cfa96cfe32cc083243: This TxnStatusTablet replica's current role is: LEADER
I20260812 21:04:06.300870 25632 catalog_manager.cc:5495] T e6c0e1f4e69c4b7ba464c3442238084f P 0d68d6ebf5ac47cfa96cfe32cc083243 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0d68d6ebf5ac47cfa96cfe32cc083243 (127.11.254.129). New cstate: current_term: 1 leader_uuid: "0d68d6ebf5ac47cfa96cfe32cc083243" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d68d6ebf5ac47cfa96cfe32cc083243" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 36765 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "868e98cd6025463aa10e6b78dff569a7" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 35013 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6abfe635d04e4e0d85de54a7c7efce63" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 35529 } health_report { overall_health: UNKNOWN } } }
I20260812 21:04:06.301170 26547 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260812 21:04:06.301265 26547 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260812 21:04:06.315840 26478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:06.313067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35820 (local address 127.11.254.129:36765)
0812 21:04:06.313108 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0812 21:04:06.313110 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:04:06.313150 (+ 40us) server_negotiation.cc:407] Connection header received
0812 21:04:06.313249 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:04:06.313250 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:04:06.313268 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:04:06.313323 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 21:04:06.313443 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:06.313926 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:06.315061 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:06.315285 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:06.315342 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:06.315538 (+ 196us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 21:04:06.315643 (+ 105us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 21:04:06.315695 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0812 21:04:06.315758 (+ 63us) server_negotiation.cc:299] Negotiation successful
0812 21:04:06.315777 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 21:04:06.315897 26536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:06.313017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.129:36765 (local address 127.0.0.1:35820)
0812 21:04:06.313105 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0812 21:04:06.313116 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 21:04:06.313171 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:04:06.313327 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:04:06.313328 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:04:06.313329 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 21:04:06.313388 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:06.313390 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:06.314037 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:06.314038 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:06.314981 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:06.314984 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:06.315354 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:06.315355 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:06.315425 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:06.315447 (+ 22us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 21:04:06.315698 (+ 251us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 21:04:06.315703 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 21:04:06.315753 (+ 50us) client_negotiation.cc:240] Negotiation successful
0812 21:04:06.315804 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49}
I20260812 21:04:06.316982 26358 raft_consensus.cc:1261] T e6c0e1f4e69c4b7ba464c3442238084f P 6abfe635d04e4e0d85de54a7c7efce63 [term 1 FOLLOWER]: Refusing update from remote peer 0d68d6ebf5ac47cfa96cfe32cc083243: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 21:04:06.317111 26130 raft_consensus.cc:1261] T e6c0e1f4e69c4b7ba464c3442238084f P 868e98cd6025463aa10e6b78dff569a7 [term 1 FOLLOWER]: Refusing update from remote peer 0d68d6ebf5ac47cfa96cfe32cc083243: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 21:04:06.317243 26544 consensus_queue.cc:1035] T e6c0e1f4e69c4b7ba464c3442238084f P 0d68d6ebf5ac47cfa96cfe32cc083243 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6abfe635d04e4e0d85de54a7c7efce63" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 35529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 21:04:06.317399 26489 consensus_queue.cc:1035] T e6c0e1f4e69c4b7ba464c3442238084f P 0d68d6ebf5ac47cfa96cfe32cc083243 [LEADER]: Connected to new peer: Peer: permanent_uuid: "868e98cd6025463aa10e6b78dff569a7" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 35013 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 21:04:06.318091 26490 tablet_replica.cc:439] T e6c0e1f4e69c4b7ba464c3442238084f P 868e98cd6025463aa10e6b78dff569a7: TxnStatusTablet state changed. Reason: New leader 0d68d6ebf5ac47cfa96cfe32cc083243. Latest consensus state: current_term: 1 leader_uuid: "0d68d6ebf5ac47cfa96cfe32cc083243" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d68d6ebf5ac47cfa96cfe32cc083243" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 36765 } } peers { permanent_uuid: "868e98cd6025463aa10e6b78dff569a7" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 35013 } } peers { permanent_uuid: "6abfe635d04e4e0d85de54a7c7efce63" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 35529 } } }
I20260812 21:04:06.318113 26491 tablet_replica.cc:439] T e6c0e1f4e69c4b7ba464c3442238084f P 6abfe635d04e4e0d85de54a7c7efce63: TxnStatusTablet state changed. Reason: New leader 0d68d6ebf5ac47cfa96cfe32cc083243. Latest consensus state: current_term: 1 leader_uuid: "0d68d6ebf5ac47cfa96cfe32cc083243" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d68d6ebf5ac47cfa96cfe32cc083243" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 36765 } } peers { permanent_uuid: "868e98cd6025463aa10e6b78dff569a7" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 35013 } } peers { permanent_uuid: "6abfe635d04e4e0d85de54a7c7efce63" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 35529 } } }
I20260812 21:04:06.318186 26490 tablet_replica.cc:442] T e6c0e1f4e69c4b7ba464c3442238084f P 868e98cd6025463aa10e6b78dff569a7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 21:04:06.318202 26491 tablet_replica.cc:442] T e6c0e1f4e69c4b7ba464c3442238084f P 6abfe635d04e4e0d85de54a7c7efce63: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 21:04:06.318495 26489 tablet_replica.cc:439] T e6c0e1f4e69c4b7ba464c3442238084f P 0d68d6ebf5ac47cfa96cfe32cc083243: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0d68d6ebf5ac47cfa96cfe32cc083243" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d68d6ebf5ac47cfa96cfe32cc083243" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 36765 } } peers { permanent_uuid: "868e98cd6025463aa10e6b78dff569a7" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 35013 } } peers { permanent_uuid: "6abfe635d04e4e0d85de54a7c7efce63" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 35529 } } }
I20260812 21:04:06.318574 26489 tablet_replica.cc:442] T e6c0e1f4e69c4b7ba464c3442238084f P 0d68d6ebf5ac47cfa96cfe32cc083243: This TxnStatusTablet replica's current role is: LEADER
I20260812 21:04:06.318658 26503 mvcc.cc:205] Tried to move back new op lower bound from 7317785175312355328 to 7317785175245701120. Current Snapshot: MvccSnapshot[applied={T|T < 7317785175312355328 or (T in {7317785175312355328})}]
I20260812 21:04:06.319062 26504 mvcc.cc:205] Tried to move back new op lower bound from 7317785175312355328 to 7317785175245701120. Current Snapshot: MvccSnapshot[applied={T|T < 7317785175312355328 or (T in {7317785175312355328})}]
I20260812 21:04:06.319166 26491 tablet_replica.cc:439] T e6c0e1f4e69c4b7ba464c3442238084f P 6abfe635d04e4e0d85de54a7c7efce63: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0d68d6ebf5ac47cfa96cfe32cc083243" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d68d6ebf5ac47cfa96cfe32cc083243" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 36765 } } peers { permanent_uuid: "868e98cd6025463aa10e6b78dff569a7" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 35013 } } peers { permanent_uuid: "6abfe635d04e4e0d85de54a7c7efce63" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 35529 } } }
I20260812 21:04:06.319236 26491 tablet_replica.cc:442] T e6c0e1f4e69c4b7ba464c3442238084f P 6abfe635d04e4e0d85de54a7c7efce63: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 21:04:06.319895 26544 tablet_replica.cc:439] T e6c0e1f4e69c4b7ba464c3442238084f P 0d68d6ebf5ac47cfa96cfe32cc083243: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0d68d6ebf5ac47cfa96cfe32cc083243" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d68d6ebf5ac47cfa96cfe32cc083243" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 36765 } } peers { permanent_uuid: "868e98cd6025463aa10e6b78dff569a7" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 35013 } } peers { permanent_uuid: "6abfe635d04e4e0d85de54a7c7efce63" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 35529 } } }
I20260812 21:04:06.319978 26544 tablet_replica.cc:442] T e6c0e1f4e69c4b7ba464c3442238084f P 0d68d6ebf5ac47cfa96cfe32cc083243: This TxnStatusTablet replica's current role is: LEADER
I20260812 21:04:06.320294 26490 tablet_replica.cc:439] T e6c0e1f4e69c4b7ba464c3442238084f P 868e98cd6025463aa10e6b78dff569a7: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0d68d6ebf5ac47cfa96cfe32cc083243" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d68d6ebf5ac47cfa96cfe32cc083243" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 36765 } } peers { permanent_uuid: "868e98cd6025463aa10e6b78dff569a7" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 35013 } } peers { permanent_uuid: "6abfe635d04e4e0d85de54a7c7efce63" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 35529 } } }
I20260812 21:04:06.320379 26490 tablet_replica.cc:442] T e6c0e1f4e69c4b7ba464c3442238084f P 868e98cd6025463aa10e6b78dff569a7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 21:04:06.325032 26478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:06.322486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.130:39601 (local address 127.11.254.129:36765)
0812 21:04:06.322522 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0812 21:04:06.322524 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:04:06.322663 (+ 139us) server_negotiation.cc:407] Connection header received
0812 21:04:06.322725 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:04:06.322726 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:04:06.322733 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:04:06.322778 (+ 45us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 21:04:06.322885 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:06.323351 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:06.324461 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:06.324647 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:06.324699 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:06.324849 (+ 150us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 21:04:06.324873 (+ 24us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 21:04:06.324903 (+ 30us) server_negotiation.cc:1035] Waiting for connection context
0812 21:04:06.324967 (+ 64us) server_negotiation.cc:299] Negotiation successful
0812 21:04:06.324982 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260812 21:04:06.325098 26564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:06.322433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.129:36765 (local address 127.11.254.130:39601)
0812 21:04:06.322590 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0812 21:04:06.322601 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 21:04:06.322672 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:04:06.322781 (+ 109us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:04:06.322782 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:04:06.322784 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 21:04:06.322835 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:06.322836 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:06.323417 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:06.323417 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:06.324389 (+ 972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:06.324392 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:06.324710 (+ 318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:06.324711 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:06.324768 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:06.324784 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 21:04:06.324917 (+ 133us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 21:04:06.324918 (+ 1us) client_negotiation.cc:769] Sending connection context
0812 21:04:06.324965 (+ 47us) client_negotiation.cc:240] Negotiation successful
0812 21:04:06.325014 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":75,"threads_started":1}
I20260812 21:04:06.327070 26358 raft_consensus.cc:1261] T 8ce6a346634047758996dda6db22ac65 P 6abfe635d04e4e0d85de54a7c7efce63 [term 1 FOLLOWER]: Refusing update from remote peer 868e98cd6025463aa10e6b78dff569a7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 21:04:06.327108 25911 raft_consensus.cc:1261] T 8ce6a346634047758996dda6db22ac65 P 0d68d6ebf5ac47cfa96cfe32cc083243 [term 1 FOLLOWER]: Refusing update from remote peer 868e98cd6025463aa10e6b78dff569a7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 21:04:06.327315 26494 consensus_queue.cc:1035] T 8ce6a346634047758996dda6db22ac65 P 868e98cd6025463aa10e6b78dff569a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6abfe635d04e4e0d85de54a7c7efce63" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 35529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 21:04:06.327437 26490 consensus_queue.cc:1035] T 8ce6a346634047758996dda6db22ac65 P 868e98cd6025463aa10e6b78dff569a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d68d6ebf5ac47cfa96cfe32cc083243" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 36765 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 21:04:06.335798 26476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:06.332985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.129:38719 (local address 127.11.254.130:35013)
0812 21:04:06.333026 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0812 21:04:06.333028 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:04:06.333259 (+ 231us) server_negotiation.cc:407] Connection header received
0812 21:04:06.333316 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:04:06.333317 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:04:06.333326 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:04:06.333377 (+ 51us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 21:04:06.333510 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:06.334060 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:06.335087 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:06.335294 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:06.335351 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:06.335547 (+ 196us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 21:04:06.335585 (+ 38us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 21:04:06.335634 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0812 21:04:06.335687 (+ 53us) server_negotiation.cc:299] Negotiation successful
0812 21:04:06.335712 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 21:04:06.335835 26576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:06.332960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.130:35013 (local address 127.11.254.129:38719)
0812 21:04:06.333191 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0812 21:04:06.333201 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 21:04:06.333257 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:04:06.333380 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:04:06.333381 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:04:06.333382 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 21:04:06.333449 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:06.333451 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:06.334135 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:06.334136 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:06.335028 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:06.335031 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:06.335372 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:06.335373 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:06.335440 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:06.335456 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 21:04:06.335645 (+ 189us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 21:04:06.335647 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 21:04:06.335686 (+ 39us) client_negotiation.cc:240] Negotiation successful
0812 21:04:06.335731 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":141,"threads_started":1}
I20260812 21:04:06.344501 25633 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58282:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 21:04:06.344619 25633 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58282:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 21:04:06.345227 25633 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5eb7e77e033b4ec48c7dabe9b94175eb: Sending DeleteTablet for 3 replicas of tablet 8ce6a346634047758996dda6db22ac65
I20260812 21:04:06.345467 25880 tablet_service.cc:1514] Processing DeleteTablet for tablet 8ce6a346634047758996dda6db22ac65 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:04:06 UTC) from {username='oryx', principal='oryx/127.11.254.190@KRBTEST.COM'} at 127.0.0.1:35804
I20260812 21:04:06.345530 26327 tablet_service.cc:1514] Processing DeleteTablet for tablet 8ce6a346634047758996dda6db22ac65 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:04:06 UTC) from {username='oryx', principal='oryx/127.11.254.190@KRBTEST.COM'} at 127.0.0.1:56292
I20260812 21:04:06.345543 26097 tablet_service.cc:1514] Processing DeleteTablet for tablet 8ce6a346634047758996dda6db22ac65 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:04:06 UTC) from {username='oryx', principal='oryx/127.11.254.190@KRBTEST.COM'} at 127.0.0.1:58964
I20260812 21:04:06.345618 26519 tablet_replica.cc:330] T 8ce6a346634047758996dda6db22ac65 P 6abfe635d04e4e0d85de54a7c7efce63: stopping tablet replica
I20260812 21:04:06.345667 26519 raft_consensus.cc:2229] T 8ce6a346634047758996dda6db22ac65 P 6abfe635d04e4e0d85de54a7c7efce63 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 21:04:06.345669 26517 tablet_replica.cc:330] T 8ce6a346634047758996dda6db22ac65 P 868e98cd6025463aa10e6b78dff569a7: stopping tablet replica
I20260812 21:04:06.345670 26516 tablet_replica.cc:330] T 8ce6a346634047758996dda6db22ac65 P 0d68d6ebf5ac47cfa96cfe32cc083243: stopping tablet replica
I20260812 21:04:06.345703 26519 raft_consensus.cc:2258] T 8ce6a346634047758996dda6db22ac65 P 6abfe635d04e4e0d85de54a7c7efce63 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 21:04:06.345723 26517 raft_consensus.cc:2229] T 8ce6a346634047758996dda6db22ac65 P 868e98cd6025463aa10e6b78dff569a7 [term 1 LEADER]: Raft consensus shutting down.
I20260812 21:04:06.345726 26516 raft_consensus.cc:2229] T 8ce6a346634047758996dda6db22ac65 P 0d68d6ebf5ac47cfa96cfe32cc083243 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 21:04:06.345762 26516 raft_consensus.cc:2258] T 8ce6a346634047758996dda6db22ac65 P 0d68d6ebf5ac47cfa96cfe32cc083243 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 21:04:06.345779 26517 raft_consensus.cc:2258] T 8ce6a346634047758996dda6db22ac65 P 868e98cd6025463aa10e6b78dff569a7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 21:04:06.346218 26517 ts_tablet_manager.cc:1905] T 8ce6a346634047758996dda6db22ac65 P 868e98cd6025463aa10e6b78dff569a7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 21:04:06.346215 26519 ts_tablet_manager.cc:1905] T 8ce6a346634047758996dda6db22ac65 P 6abfe635d04e4e0d85de54a7c7efce63: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 21:04:06.346525 26516 ts_tablet_manager.cc:1905] T 8ce6a346634047758996dda6db22ac65 P 0d68d6ebf5ac47cfa96cfe32cc083243: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 21:04:06.347924 26519 ts_tablet_manager.cc:1918] T 8ce6a346634047758996dda6db22ac65 P 6abfe635d04e4e0d85de54a7c7efce63: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 21:04:06.347934 26517 ts_tablet_manager.cc:1918] T 8ce6a346634047758996dda6db22ac65 P 868e98cd6025463aa10e6b78dff569a7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 21:04:06.347980 26517 log.cc:1196] T 8ce6a346634047758996dda6db22ac65 P 868e98cd6025463aa10e6b78dff569a7: Deleting WAL directory at /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568607354921-12282-0/minicluster-data/ts-1/wal/wals/8ce6a346634047758996dda6db22ac65
I20260812 21:04:06.347980 26519 log.cc:1196] T 8ce6a346634047758996dda6db22ac65 P 6abfe635d04e4e0d85de54a7c7efce63: Deleting WAL directory at /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568607354921-12282-0/minicluster-data/ts-2/wal/wals/8ce6a346634047758996dda6db22ac65
I20260812 21:04:06.348060 26516 ts_tablet_manager.cc:1918] T 8ce6a346634047758996dda6db22ac65 P 0d68d6ebf5ac47cfa96cfe32cc083243: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 21:04:06.348116 26516 log.cc:1196] T 8ce6a346634047758996dda6db22ac65 P 0d68d6ebf5ac47cfa96cfe32cc083243: Deleting WAL directory at /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568607354921-12282-0/minicluster-data/ts-0/wal/wals/8ce6a346634047758996dda6db22ac65
I20260812 21:04:06.348192 26519 ts_tablet_manager.cc:1939] T 8ce6a346634047758996dda6db22ac65 P 6abfe635d04e4e0d85de54a7c7efce63: Deleting consensus metadata
I20260812 21:04:06.348214 26517 ts_tablet_manager.cc:1939] T 8ce6a346634047758996dda6db22ac65 P 868e98cd6025463aa10e6b78dff569a7: Deleting consensus metadata
I20260812 21:04:06.348332 26516 ts_tablet_manager.cc:1939] T 8ce6a346634047758996dda6db22ac65 P 0d68d6ebf5ac47cfa96cfe32cc083243: Deleting consensus metadata
I20260812 21:04:06.348599 25617 catalog_manager.cc:4841] TS 868e98cd6025463aa10e6b78dff569a7 (127.11.254.130:35013): tablet 8ce6a346634047758996dda6db22ac65 (table test-table [id=4427eb72b7d542238e5d5b4e6f4ff490]) successfully deleted
I20260812 21:04:06.348637 25613 catalog_manager.cc:4841] TS 0d68d6ebf5ac47cfa96cfe32cc083243 (127.11.254.129:36765): tablet 8ce6a346634047758996dda6db22ac65 (table test-table [id=4427eb72b7d542238e5d5b4e6f4ff490]) successfully deleted
I20260812 21:04:06.348847 25615 catalog_manager.cc:4841] TS 6abfe635d04e4e0d85de54a7c7efce63 (127.11.254.131:35529): tablet 8ce6a346634047758996dda6db22ac65 (table test-table [id=4427eb72b7d542238e5d5b4e6f4ff490]) successfully deleted
Aug 12 21:04:06 dist-test-slave-vpvm krb5kdc[25571](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568646, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 21:04:06 dist-test-slave-vpvm krb5kdc[25571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.11.254.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 21:04:06 dist-test-slave-vpvm krb5kdc[25571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.11.254.190@KRBTEST.COM, Server not found in Kerberos database
W20260812 21:04:06.372767 25724 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:04:06.366423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58302 (local address 127.11.254.190:43839)
0812 21:04:06.366483 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0812 21:04:06.366485 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:04:06.366627 (+ 142us) server_negotiation.cc:407] Connection header received
0812 21:04:06.366706 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:04:06.366707 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:04:06.366718 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:04:06.366768 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:04:06.367455 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:06.367966 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:06.369055 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:06.369262 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:06.369320 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:06.372697 (+ 3377us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58302: BlockingRecv error: recv got EOF from 127.0.0.1:58302 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
Aug 12 21:04:06 dist-test-slave-vpvm krb5kdc[25571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568646, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.11.254.190@KRBTEST.COM
I20260812 21:04:06.381072 25724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:06.373729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58304 (local address 127.11.254.190:43839)
0812 21:04:06.373769 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0812 21:04:06.373771 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:04:06.373783 (+ 12us) server_negotiation.cc:407] Connection header received
0812 21:04:06.373809 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:04:06.373809 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:04:06.373817 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:04:06.373862 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:04:06.374442 (+ 580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:06.374916 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:06.375882 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:06.376066 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:06.376115 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:06.378484 (+ 2369us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:04:06.378486 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:04:06.378486 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:04:06.378494 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:04:06.379781 (+ 1287us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:04:06.380158 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:04:06.380159 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:04:06.380159 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:04:06.380228 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:04:06.380477 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:04:06.380478 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:04:06.380479 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:04:06.380640 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:04:06.380740 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0812 21:04:06.380867 (+ 127us) server_negotiation.cc:299] Negotiation successful
0812 21:04:06.380952 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260812 21:04:06.382736 25633 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58304:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 21:04:06.382910 25633 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 21:04:06.384270 25880 tablet_service.cc:1467] Processing CreateTablet for tablet 562bf132b97146449f20cb8ee647fb51 (DEFAULT_TABLE table=test-table [id=1cf7384515c4483a90b33b86693386bc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:04:06.384290 26097 tablet_service.cc:1467] Processing CreateTablet for tablet 562bf132b97146449f20cb8ee647fb51 (DEFAULT_TABLE table=test-table [id=1cf7384515c4483a90b33b86693386bc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:04:06.384414 25880 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 562bf132b97146449f20cb8ee647fb51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:04:06.384433 26097 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 562bf132b97146449f20cb8ee647fb51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:04:06.385649 26327 tablet_service.cc:1467] Processing CreateTablet for tablet 562bf132b97146449f20cb8ee647fb51 (DEFAULT_TABLE table=test-table [id=1cf7384515c4483a90b33b86693386bc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:04:06.385785 26327 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 562bf132b97146449f20cb8ee647fb51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:04:06.385818 26479 tablet_bootstrap.cc:492] T 562bf132b97146449f20cb8ee647fb51 P 0d68d6ebf5ac47cfa96cfe32cc083243: Bootstrap starting.
I20260812 21:04:06.386050 26484 tablet_bootstrap.cc:492] T 562bf132b97146449f20cb8ee647fb51 P 868e98cd6025463aa10e6b78dff569a7: Bootstrap starting.
I20260812 21:04:06.386401 26484 tablet_bootstrap.cc:654] T 562bf132b97146449f20cb8ee647fb51 P 868e98cd6025463aa10e6b78dff569a7: Neither blocks nor log segments found. Creating new log.
I20260812 21:04:06.387006 26481 tablet_bootstrap.cc:492] T 562bf132b97146449f20cb8ee647fb51 P 6abfe635d04e4e0d85de54a7c7efce63: Bootstrap starting.
I20260812 21:04:06.387115 26479 tablet_bootstrap.cc:654] T 562bf132b97146449f20cb8ee647fb51 P 0d68d6ebf5ac47cfa96cfe32cc083243: Neither blocks nor log segments found. Creating new log.
I20260812 21:04:06.387454 26481 tablet_bootstrap.cc:654] T 562bf132b97146449f20cb8ee647fb51 P 6abfe635d04e4e0d85de54a7c7efce63: Neither blocks nor log segments found. Creating new log.
I20260812 21:04:06.387774 26484 tablet_bootstrap.cc:492] T 562bf132b97146449f20cb8ee647fb51 P 868e98cd6025463aa10e6b78dff569a7: No bootstrap required, opened a new log
I20260812 21:04:06.387820 26484 ts_tablet_manager.cc:1397] T 562bf132b97146449f20cb8ee647fb51 P 868e98cd6025463aa10e6b78dff569a7: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260812 21:04:06.387846 26479 tablet_bootstrap.cc:492] T 562bf132b97146449f20cb8ee647fb51 P 0d68d6ebf5ac47cfa96cfe32cc083243: No bootstrap required, opened a new log
I20260812 21:04:06.387889 26479 ts_tablet_manager.cc:1397] T 562bf132b97146449f20cb8ee647fb51 P 0d68d6ebf5ac47cfa96cfe32cc083243: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260812 21:04:06.387949 26484 raft_consensus.cc:348] T 562bf132b97146449f20cb8ee647fb51 P 868e98cd6025463aa10e6b78dff569a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d68d6ebf5ac47cfa96cfe32cc083243" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 36765 } } peers { permanent_uuid: "868e98cd6025463aa10e6b78dff569a7" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 35013 } } peers { permanent_uuid: "6abfe635d04e4e0d85de54a7c7efce63" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 35529 } }
I20260812 21:04:06.388032 26484 raft_consensus.cc:374] T 562bf132b97146449f20cb8ee647fb51 P 868e98cd6025463aa10e6b78dff569a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:04:06.388051 26484 raft_consensus.cc:729] T 562bf132b97146449f20cb8ee647fb51 P 868e98cd6025463aa10e6b78dff569a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 868e98cd6025463aa10e6b78dff569a7, State: Initialized, Role: FOLLOWER
I20260812 21:04:06.388093 26484 consensus_queue.cc:260] T 562bf132b97146449f20cb8ee647fb51 P 868e98cd6025463aa10e6b78dff569a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d68d6ebf5ac47cfa96cfe32cc083243" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 36765 } } peers { permanent_uuid: "868e98cd6025463aa10e6b78dff569a7" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 35013 } } peers { permanent_uuid: "6abfe635d04e4e0d85de54a7c7efce63" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 35529 } }
I20260812 21:04:06.388052 26479 raft_consensus.cc:348] T 562bf132b97146449f20cb8ee647fb51 P 0d68d6ebf5ac47cfa96cfe32cc083243 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d68d6ebf5ac47cfa96cfe32cc083243" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 36765 } } peers { permanent_uuid: "868e98cd6025463aa10e6b78dff569a7" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 35013 } } peers { permanent_uuid: "6abfe635d04e4e0d85de54a7c7efce63" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 35529 } }
I20260812 21:04:06.388141 26479 raft_consensus.cc:374] T 562bf132b97146449f20cb8ee647fb51 P 0d68d6ebf5ac47cfa96cfe32cc083243 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:04:06.388168 26479 raft_consensus.cc:729] T 562bf132b97146449f20cb8ee647fb51 P 0d68d6ebf5ac47cfa96cfe32cc083243 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d68d6ebf5ac47cfa96cfe32cc083243, State: Initialized, Role: FOLLOWER
I20260812 21:04:06.388183 26484 ts_tablet_manager.cc:1428] T 562bf132b97146449f20cb8ee647fb51 P 868e98cd6025463aa10e6b78dff569a7: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260812 21:04:06.388216 26479 consensus_queue.cc:260] T 562bf132b97146449f20cb8ee647fb51 P 0d68d6ebf5ac47cfa96cfe32cc083243 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d68d6ebf5ac47cfa96cfe32cc083243" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 36765 } } peers { permanent_uuid: "868e98cd6025463aa10e6b78dff569a7" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 35013 } } peers { permanent_uuid: "6abfe635d04e4e0d85de54a7c7efce63" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 35529 } }
I20260812 21:04:06.388331 26479 ts_tablet_manager.cc:1428] T 562bf132b97146449f20cb8ee647fb51 P 0d68d6ebf5ac47cfa96cfe32cc083243: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 21:04:06.389142 26481 tablet_bootstrap.cc:492] T 562bf132b97146449f20cb8ee647fb51 P 6abfe635d04e4e0d85de54a7c7efce63: No bootstrap required, opened a new log
I20260812 21:04:06.389191 26481 ts_tablet_manager.cc:1397] T 562bf132b97146449f20cb8ee647fb51 P 6abfe635d04e4e0d85de54a7c7efce63: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 21:04:06.389334 26481 raft_consensus.cc:348] T 562bf132b97146449f20cb8ee647fb51 P 6abfe635d04e4e0d85de54a7c7efce63 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d68d6ebf5ac47cfa96cfe32cc083243" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 36765 } } peers { permanent_uuid: "868e98cd6025463aa10e6b78dff569a7" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 35013 } } peers { permanent_uuid: "6abfe635d04e4e0d85de54a7c7efce63" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 35529 } }
I20260812 21:04:06.389393 26481 raft_consensus.cc:374] T 562bf132b97146449f20cb8ee647fb51 P 6abfe635d04e4e0d85de54a7c7efce63 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:04:06.389410 26481 raft_consensus.cc:729] T 562bf132b97146449f20cb8ee647fb51 P 6abfe635d04e4e0d85de54a7c7efce63 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6abfe635d04e4e0d85de54a7c7efce63, State: Initialized, Role: FOLLOWER
I20260812 21:04:06.389473 26481 consensus_queue.cc:260] T 562bf132b97146449f20cb8ee647fb51 P 6abfe635d04e4e0d85de54a7c7efce63 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d68d6ebf5ac47cfa96cfe32cc083243" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 36765 } } peers { permanent_uuid: "868e98cd6025463aa10e6b78dff569a7" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 35013 } } peers { permanent_uuid: "6abfe635d04e4e0d85de54a7c7efce63" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 35529 } }
I20260812 21:04:06.389576 26481 ts_tablet_manager.cc:1428] T 562bf132b97146449f20cb8ee647fb51 P 6abfe635d04e4e0d85de54a7c7efce63: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260812 21:04:06.443521 26568 raft_consensus.cc:482] T 562bf132b97146449f20cb8ee647fb51 P 0d68d6ebf5ac47cfa96cfe32cc083243 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 21:04:06.443634 26568 raft_consensus.cc:504] T 562bf132b97146449f20cb8ee647fb51 P 0d68d6ebf5ac47cfa96cfe32cc083243 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d68d6ebf5ac47cfa96cfe32cc083243" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 36765 } } peers { permanent_uuid: "868e98cd6025463aa10e6b78dff569a7" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 35013 } } peers { permanent_uuid: "6abfe635d04e4e0d85de54a7c7efce63" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 35529 } }
I20260812 21:04:06.443810 26568 leader_election.cc:288] T 562bf132b97146449f20cb8ee647fb51 P 0d68d6ebf5ac47cfa96cfe32cc083243 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 868e98cd6025463aa10e6b78dff569a7 (127.11.254.130:35013), 6abfe635d04e4e0d85de54a7c7efce63 (127.11.254.131:35529)
I20260812 21:04:06.444001 26130 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "562bf132b97146449f20cb8ee647fb51" candidate_uuid: "0d68d6ebf5ac47cfa96cfe32cc083243" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "868e98cd6025463aa10e6b78dff569a7" is_pre_election: true
I20260812 21:04:06.444058 26358 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "562bf132b97146449f20cb8ee647fb51" candidate_uuid: "0d68d6ebf5ac47cfa96cfe32cc083243" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6abfe635d04e4e0d85de54a7c7efce63" is_pre_election: true
I20260812 21:04:06.444132 26130 raft_consensus.cc:2454] T 562bf132b97146449f20cb8ee647fb51 P 868e98cd6025463aa10e6b78dff569a7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d68d6ebf5ac47cfa96cfe32cc083243 in term 0.
I20260812 21:04:06.444142 26358 raft_consensus.cc:2454] T 562bf132b97146449f20cb8ee647fb51 P 6abfe635d04e4e0d85de54a7c7efce63 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d68d6ebf5ac47cfa96cfe32cc083243 in term 0.
I20260812 21:04:06.444300 25793 leader_election.cc:302] T 562bf132b97146449f20cb8ee647fb51 P 0d68d6ebf5ac47cfa96cfe32cc083243 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d68d6ebf5ac47cfa96cfe32cc083243, 6abfe635d04e4e0d85de54a7c7efce63; no voters:
I20260812 21:04:06.444408 26568 raft_consensus.cc:2781] T 562bf132b97146449f20cb8ee647fb51 P 0d68d6ebf5ac47cfa96cfe32cc083243 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 21:04:06.444443 26568 raft_consensus.cc:482] T 562bf132b97146449f20cb8ee647fb51 P 0d68d6ebf5ac47cfa96cfe32cc083243 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 21:04:06.444463 26568 raft_consensus.cc:3037] T 562bf132b97146449f20cb8ee647fb51 P 0d68d6ebf5ac47cfa96cfe32cc083243 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:04:06.445109 26568 raft_consensus.cc:504] T 562bf132b97146449f20cb8ee647fb51 P 0d68d6ebf5ac47cfa96cfe32cc083243 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d68d6ebf5ac47cfa96cfe32cc083243" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 36765 } } peers { permanent_uuid: "868e98cd6025463aa10e6b78dff569a7" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 35013 } } peers { permanent_uuid: "6abfe635d04e4e0d85de54a7c7efce63" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 35529 } }
I20260812 21:04:06.445235 26568 leader_election.cc:288] T 562bf132b97146449f20cb8ee647fb51 P 0d68d6ebf5ac47cfa96cfe32cc083243 [CANDIDATE]: Term 1 election: Requested vote from peers 868e98cd6025463aa10e6b78dff569a7 (127.11.254.130:35013), 6abfe635d04e4e0d85de54a7c7efce63 (127.11.254.131:35529)
I20260812 21:04:06.445386 26130 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "562bf132b97146449f20cb8ee647fb51" candidate_uuid: "0d68d6ebf5ac47cfa96cfe32cc083243" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "868e98cd6025463aa10e6b78dff569a7"
I20260812 21:04:06.445400 26358 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "562bf132b97146449f20cb8ee647fb51" candidate_uuid: "0d68d6ebf5ac47cfa96cfe32cc083243" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6abfe635d04e4e0d85de54a7c7efce63"
I20260812 21:04:06.445475 26358 raft_consensus.cc:3037] T 562bf132b97146449f20cb8ee647fb51 P 6abfe635d04e4e0d85de54a7c7efce63 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:04:06.445475 26130 raft_consensus.cc:3037] T 562bf132b97146449f20cb8ee647fb51 P 868e98cd6025463aa10e6b78dff569a7 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:04:06.446203 26358 raft_consensus.cc:2454] T 562bf132b97146449f20cb8ee647fb51 P 6abfe635d04e4e0d85de54a7c7efce63 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d68d6ebf5ac47cfa96cfe32cc083243 in term 1.
I20260812 21:04:06.446321 26130 raft_consensus.cc:2454] T 562bf132b97146449f20cb8ee647fb51 P 868e98cd6025463aa10e6b78dff569a7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d68d6ebf5ac47cfa96cfe32cc083243 in term 1.
I20260812 21:04:06.446522 25797 leader_election.cc:302] T 562bf132b97146449f20cb8ee647fb51 P 0d68d6ebf5ac47cfa96cfe32cc083243 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d68d6ebf5ac47cfa96cfe32cc083243, 868e98cd6025463aa10e6b78dff569a7; no voters:
I20260812 21:04:06.446632 26568 raft_consensus.cc:2781] T 562bf132b97146449f20cb8ee647fb51 P 0d68d6ebf5ac47cfa96cfe32cc083243 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:04:06.446702 26568 raft_consensus.cc:686] T 562bf132b97146449f20cb8ee647fb51 P 0d68d6ebf5ac47cfa96cfe32cc083243 [term 1 LEADER]: Becoming Leader. State: Replica: 0d68d6ebf5ac47cfa96cfe32cc083243, State: Running, Role: LEADER
I20260812 21:04:06.446748 26568 consensus_queue.cc:237] T 562bf132b97146449f20cb8ee647fb51 P 0d68d6ebf5ac47cfa96cfe32cc083243 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d68d6ebf5ac47cfa96cfe32cc083243" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 36765 } } peers { permanent_uuid: "868e98cd6025463aa10e6b78dff569a7" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 35013 } } peers { permanent_uuid: "6abfe635d04e4e0d85de54a7c7efce63" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 35529 } }
I20260812 21:04:06.448355 25633 catalog_manager.cc:5495] T 562bf132b97146449f20cb8ee647fb51 P 0d68d6ebf5ac47cfa96cfe32cc083243 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0d68d6ebf5ac47cfa96cfe32cc083243 (127.11.254.129). New cstate: current_term: 1 leader_uuid: "0d68d6ebf5ac47cfa96cfe32cc083243" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d68d6ebf5ac47cfa96cfe32cc083243" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 36765 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "868e98cd6025463aa10e6b78dff569a7" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 35013 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6abfe635d04e4e0d85de54a7c7efce63" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 35529 } health_report { overall_health: UNKNOWN } } }
I20260812 21:04:06.461592 26478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:06.459462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35830 (local address 127.11.254.129:36765)
0812 21:04:06.459526 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0812 21:04:06.459528 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:04:06.459537 (+ 9us) server_negotiation.cc:407] Connection header received
0812 21:04:06.459560 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:04:06.459561 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:04:06.459571 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:04:06.459614 (+ 43us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 21:04:06.459775 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:06.460246 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:06.460977 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:06.461169 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:06.461225 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:06.461372 (+ 147us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 21:04:06.461401 (+ 29us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 21:04:06.461441 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0812 21:04:06.461489 (+ 48us) server_negotiation.cc:299] Negotiation successful
0812 21:04:06.461513 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260812 21:04:06.462409 26358 raft_consensus.cc:1261] T 562bf132b97146449f20cb8ee647fb51 P 6abfe635d04e4e0d85de54a7c7efce63 [term 1 FOLLOWER]: Refusing update from remote peer 0d68d6ebf5ac47cfa96cfe32cc083243: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 21:04:06.462442 26130 raft_consensus.cc:1261] T 562bf132b97146449f20cb8ee647fb51 P 868e98cd6025463aa10e6b78dff569a7 [term 1 FOLLOWER]: Refusing update from remote peer 0d68d6ebf5ac47cfa96cfe32cc083243: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 21:04:06.462668 26544 consensus_queue.cc:1035] T 562bf132b97146449f20cb8ee647fb51 P 0d68d6ebf5ac47cfa96cfe32cc083243 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6abfe635d04e4e0d85de54a7c7efce63" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 35529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 21:04:06.462790 26568 consensus_queue.cc:1035] T 562bf132b97146449f20cb8ee647fb51 P 0d68d6ebf5ac47cfa96cfe32cc083243 [LEADER]: Connected to new peer: Peer: permanent_uuid: "868e98cd6025463aa10e6b78dff569a7" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 35013 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 21:04:06.465719 25631 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58304:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 21:04:06.465807 25631 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58304:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 21:04:06.466441 25631 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5eb7e77e033b4ec48c7dabe9b94175eb: Sending DeleteTablet for 3 replicas of tablet 562bf132b97146449f20cb8ee647fb51
I20260812 21:04:06.466683 26327 tablet_service.cc:1514] Processing DeleteTablet for tablet 562bf132b97146449f20cb8ee647fb51 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:04:06 UTC) from {username='oryx', principal='oryx/127.11.254.190@KRBTEST.COM'} at 127.0.0.1:56292
I20260812 21:04:06.466711 25880 tablet_service.cc:1514] Processing DeleteTablet for tablet 562bf132b97146449f20cb8ee647fb51 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:04:06 UTC) from {username='oryx', principal='oryx/127.11.254.190@KRBTEST.COM'} at 127.0.0.1:35804
I20260812 21:04:06.466799 26516 tablet_replica.cc:330] T 562bf132b97146449f20cb8ee647fb51 P 0d68d6ebf5ac47cfa96cfe32cc083243: stopping tablet replica
I20260812 21:04:06.466804 26519 tablet_replica.cc:330] T 562bf132b97146449f20cb8ee647fb51 P 6abfe635d04e4e0d85de54a7c7efce63: stopping tablet replica
I20260812 21:04:06.466854 26516 raft_consensus.cc:2229] T 562bf132b97146449f20cb8ee647fb51 P 0d68d6ebf5ac47cfa96cfe32cc083243 [term 1 LEADER]: Raft consensus shutting down.
I20260812 21:04:06.466854 26519 raft_consensus.cc:2229] T 562bf132b97146449f20cb8ee647fb51 P 6abfe635d04e4e0d85de54a7c7efce63 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 21:04:06.466904 26519 raft_consensus.cc:2258] T 562bf132b97146449f20cb8ee647fb51 P 6abfe635d04e4e0d85de54a7c7efce63 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 21:04:06.466948 26516 raft_consensus.cc:2258] T 562bf132b97146449f20cb8ee647fb51 P 0d68d6ebf5ac47cfa96cfe32cc083243 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 21:04:06.467233 26097 tablet_service.cc:1514] Processing DeleteTablet for tablet 562bf132b97146449f20cb8ee647fb51 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:04:06 UTC) from {username='oryx', principal='oryx/127.11.254.190@KRBTEST.COM'} at 127.0.0.1:58964
I20260812 21:04:06.467283 26516 ts_tablet_manager.cc:1905] T 562bf132b97146449f20cb8ee647fb51 P 0d68d6ebf5ac47cfa96cfe32cc083243: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 21:04:06.467340 26517 tablet_replica.cc:330] T 562bf132b97146449f20cb8ee647fb51 P 868e98cd6025463aa10e6b78dff569a7: stopping tablet replica
I20260812 21:04:06.467404 26517 raft_consensus.cc:2229] T 562bf132b97146449f20cb8ee647fb51 P 868e98cd6025463aa10e6b78dff569a7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 21:04:06.467449 26517 raft_consensus.cc:2258] T 562bf132b97146449f20cb8ee647fb51 P 868e98cd6025463aa10e6b78dff569a7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 21:04:06.467739 25631 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58304:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 21:04:06.467888 25631 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 21:04:06.467903 26517 ts_tablet_manager.cc:1905] T 562bf132b97146449f20cb8ee647fb51 P 868e98cd6025463aa10e6b78dff569a7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 21:04:06.468888 26516 ts_tablet_manager.cc:1918] T 562bf132b97146449f20cb8ee647fb51 P 0d68d6ebf5ac47cfa96cfe32cc083243: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 21:04:06.468946 26516 log.cc:1196] T 562bf132b97146449f20cb8ee647fb51 P 0d68d6ebf5ac47cfa96cfe32cc083243: Deleting WAL directory at /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568607354921-12282-0/minicluster-data/ts-0/wal/wals/562bf132b97146449f20cb8ee647fb51
I20260812 21:04:06.469156 26516 ts_tablet_manager.cc:1939] T 562bf132b97146449f20cb8ee647fb51 P 0d68d6ebf5ac47cfa96cfe32cc083243: Deleting consensus metadata
I20260812 21:04:06.469174 25880 tablet_service.cc:1467] Processing CreateTablet for tablet cfc3265f9d4549fba44bec6968627002 (DEFAULT_TABLE table=test-table [id=460706200b364e76a6dff796f266d011]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:04:06.469290 25880 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cfc3265f9d4549fba44bec6968627002. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:04:06.469426 26517 ts_tablet_manager.cc:1918] T 562bf132b97146449f20cb8ee647fb51 P 868e98cd6025463aa10e6b78dff569a7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 21:04:06.469473 26517 log.cc:1196] T 562bf132b97146449f20cb8ee647fb51 P 868e98cd6025463aa10e6b78dff569a7: Deleting WAL directory at /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568607354921-12282-0/minicluster-data/ts-1/wal/wals/562bf132b97146449f20cb8ee647fb51
I20260812 21:04:06.469646 26519 ts_tablet_manager.cc:1905] T 562bf132b97146449f20cb8ee647fb51 P 6abfe635d04e4e0d85de54a7c7efce63: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 21:04:06.469671 26517 ts_tablet_manager.cc:1939] T 562bf132b97146449f20cb8ee647fb51 P 868e98cd6025463aa10e6b78dff569a7: Deleting consensus metadata
I20260812 21:04:06.470504 26479 tablet_bootstrap.cc:492] T cfc3265f9d4549fba44bec6968627002 P 0d68d6ebf5ac47cfa96cfe32cc083243: Bootstrap starting.
I20260812 21:04:06.470574 25613 catalog_manager.cc:4841] TS 0d68d6ebf5ac47cfa96cfe32cc083243 (127.11.254.129:36765): tablet 562bf132b97146449f20cb8ee647fb51 (table test-table [id=1cf7384515c4483a90b33b86693386bc]) successfully deleted
I20260812 21:04:06.470724 25617 catalog_manager.cc:4841] TS 868e98cd6025463aa10e6b78dff569a7 (127.11.254.130:35013): tablet 562bf132b97146449f20cb8ee647fb51 (table test-table [id=1cf7384515c4483a90b33b86693386bc]) successfully deleted
I20260812 21:04:06.470835 26479 tablet_bootstrap.cc:654] T cfc3265f9d4549fba44bec6968627002 P 0d68d6ebf5ac47cfa96cfe32cc083243: Neither blocks nor log segments found. Creating new log.
I20260812 21:04:06.470860 26097 tablet_service.cc:1467] Processing CreateTablet for tablet cfc3265f9d4549fba44bec6968627002 (DEFAULT_TABLE table=test-table [id=460706200b364e76a6dff796f266d011]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:04:06.470978 26097 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cfc3265f9d4549fba44bec6968627002. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:04:06.471012 26519 ts_tablet_manager.cc:1918] T 562bf132b97146449f20cb8ee647fb51 P 6abfe635d04e4e0d85de54a7c7efce63: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 21:04:06.471058 26519 log.cc:1196] T 562bf132b97146449f20cb8ee647fb51 P 6abfe635d04e4e0d85de54a7c7efce63: Deleting WAL directory at /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568607354921-12282-0/minicluster-data/ts-2/wal/wals/562bf132b97146449f20cb8ee647fb51
I20260812 21:04:06.471216 26519 ts_tablet_manager.cc:1939] T 562bf132b97146449f20cb8ee647fb51 P 6abfe635d04e4e0d85de54a7c7efce63: Deleting consensus metadata
I20260812 21:04:06.471782 26479 tablet_bootstrap.cc:492] T cfc3265f9d4549fba44bec6968627002 P 0d68d6ebf5ac47cfa96cfe32cc083243: No bootstrap required, opened a new log
I20260812 21:04:06.471817 26479 ts_tablet_manager.cc:1397] T cfc3265f9d4549fba44bec6968627002 P 0d68d6ebf5ac47cfa96cfe32cc083243: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260812 21:04:06.471902 26479 raft_consensus.cc:348] T cfc3265f9d4549fba44bec6968627002 P 0d68d6ebf5ac47cfa96cfe32cc083243 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d68d6ebf5ac47cfa96cfe32cc083243" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 36765 } } peers { permanent_uuid: "868e98cd6025463aa10e6b78dff569a7" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 35013 } } peers { permanent_uuid: "6abfe635d04e4e0d85de54a7c7efce63" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 35529 } }
I20260812 21:04:06.471947 26479 raft_consensus.cc:374] T cfc3265f9d4549fba44bec6968627002 P 0d68d6ebf5ac47cfa96cfe32cc083243 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:04:06.471959 26479 raft_consensus.cc:729] T cfc3265f9d4549fba44bec6968627002 P 0d68d6ebf5ac47cfa96cfe32cc083243 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d68d6ebf5ac47cfa96cfe32cc083243, State: Initialized, Role: FOLLOWER
I20260812 21:04:06.471994 26479 consensus_queue.cc:260] T cfc3265f9d4549fba44bec6968627002 P 0d68d6ebf5ac47cfa96cfe32cc083243 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d68d6ebf5ac47cfa96cfe32cc083243" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 36765 } } peers { permanent_uuid: "868e98cd6025463aa10e6b78dff569a7" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 35013 } } peers { permanent_uuid: "6abfe635d04e4e0d85de54a7c7efce63" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 35529 } }
I20260812 21:04:06.472075 26479 ts_tablet_manager.cc:1428] T cfc3265f9d4549fba44bec6968627002 P 0d68d6ebf5ac47cfa96cfe32cc083243: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 21:04:06.472159 26484 tablet_bootstrap.cc:492] T cfc3265f9d4549fba44bec6968627002 P 868e98cd6025463aa10e6b78dff569a7: Bootstrap starting.
I20260812 21:04:06.472453 26484 tablet_bootstrap.cc:654] T cfc3265f9d4549fba44bec6968627002 P 868e98cd6025463aa10e6b78dff569a7: Neither blocks nor log segments found. Creating new log.
I20260812 21:04:06.472859 25615 catalog_manager.cc:4841] TS 6abfe635d04e4e0d85de54a7c7efce63 (127.11.254.131:35529): tablet 562bf132b97146449f20cb8ee647fb51 (table test-table [id=1cf7384515c4483a90b33b86693386bc]) successfully deleted
I20260812 21:04:06.472922 26327 tablet_service.cc:1467] Processing CreateTablet for tablet cfc3265f9d4549fba44bec6968627002 (DEFAULT_TABLE table=test-table [id=460706200b364e76a6dff796f266d011]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:04:06.473006 26327 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cfc3265f9d4549fba44bec6968627002. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:04:06.473201 26484 tablet_bootstrap.cc:492] T cfc3265f9d4549fba44bec6968627002 P 868e98cd6025463aa10e6b78dff569a7: No bootstrap required, opened a new log
I20260812 21:04:06.473242 26484 ts_tablet_manager.cc:1397] T cfc3265f9d4549fba44bec6968627002 P 868e98cd6025463aa10e6b78dff569a7: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260812 21:04:06.473335 26484 raft_consensus.cc:348] T cfc3265f9d4549fba44bec6968627002 P 868e98cd6025463aa10e6b78dff569a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d68d6ebf5ac47cfa96cfe32cc083243" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 36765 } } peers { permanent_uuid: "868e98cd6025463aa10e6b78dff569a7" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 35013 } } peers { permanent_uuid: "6abfe635d04e4e0d85de54a7c7efce63" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 35529 } }
I20260812 21:04:06.473397 26484 raft_consensus.cc:374] T cfc3265f9d4549fba44bec6968627002 P 868e98cd6025463aa10e6b78dff569a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:04:06.473415 26484 raft_consensus.cc:729] T cfc3265f9d4549fba44bec6968627002 P 868e98cd6025463aa10e6b78dff569a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 868e98cd6025463aa10e6b78dff569a7, State: Initialized, Role: FOLLOWER
I20260812 21:04:06.473460 26484 consensus_queue.cc:260] T cfc3265f9d4549fba44bec6968627002 P 868e98cd6025463aa10e6b78dff569a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d68d6ebf5ac47cfa96cfe32cc083243" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 36765 } } peers { permanent_uuid: "868e98cd6025463aa10e6b78dff569a7" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 35013 } } peers { permanent_uuid: "6abfe635d04e4e0d85de54a7c7efce63" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 35529 } }
I20260812 21:04:06.473541 26484 ts_tablet_manager.cc:1428] T cfc3265f9d4549fba44bec6968627002 P 868e98cd6025463aa10e6b78dff569a7: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 21:04:06.474217 26481 tablet_bootstrap.cc:492] T cfc3265f9d4549fba44bec6968627002 P 6abfe635d04e4e0d85de54a7c7efce63: Bootstrap starting.
I20260812 21:04:06.474857 26481 tablet_bootstrap.cc:654] T cfc3265f9d4549fba44bec6968627002 P 6abfe635d04e4e0d85de54a7c7efce63: Neither blocks nor log segments found. Creating new log.
I20260812 21:04:06.475519 26481 tablet_bootstrap.cc:492] T cfc3265f9d4549fba44bec6968627002 P 6abfe635d04e4e0d85de54a7c7efce63: No bootstrap required, opened a new log
I20260812 21:04:06.475561 26481 ts_tablet_manager.cc:1397] T cfc3265f9d4549fba44bec6968627002 P 6abfe635d04e4e0d85de54a7c7efce63: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 21:04:06.475687 26481 raft_consensus.cc:348] T cfc3265f9d4549fba44bec6968627002 P 6abfe635d04e4e0d85de54a7c7efce63 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d68d6ebf5ac47cfa96cfe32cc083243" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 36765 } } peers { permanent_uuid: "868e98cd6025463aa10e6b78dff569a7" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 35013 } } peers { permanent_uuid: "6abfe635d04e4e0d85de54a7c7efce63" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 35529 } }
I20260812 21:04:06.475749 26481 raft_consensus.cc:374] T cfc3265f9d4549fba44bec6968627002 P 6abfe635d04e4e0d85de54a7c7efce63 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:04:06.475761 26481 raft_consensus.cc:729] T cfc3265f9d4549fba44bec6968627002 P 6abfe635d04e4e0d85de54a7c7efce63 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6abfe635d04e4e0d85de54a7c7efce63, State: Initialized, Role: FOLLOWER
I20260812 21:04:06.475800 26481 consensus_queue.cc:260] T cfc3265f9d4549fba44bec6968627002 P 6abfe635d04e4e0d85de54a7c7efce63 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d68d6ebf5ac47cfa96cfe32cc083243" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 36765 } } peers { permanent_uuid: "868e98cd6025463aa10e6b78dff569a7" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 35013 } } peers { permanent_uuid: "6abfe635d04e4e0d85de54a7c7efce63" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 35529 } }
I20260812 21:04:06.475881 26481 ts_tablet_manager.cc:1428] T cfc3265f9d4549fba44bec6968627002 P 6abfe635d04e4e0d85de54a7c7efce63: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 21:04:06.546906 26491 raft_consensus.cc:482] T cfc3265f9d4549fba44bec6968627002 P 6abfe635d04e4e0d85de54a7c7efce63 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 21:04:06.547003 26491 raft_consensus.cc:504] T cfc3265f9d4549fba44bec6968627002 P 6abfe635d04e4e0d85de54a7c7efce63 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d68d6ebf5ac47cfa96cfe32cc083243" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 36765 } } peers { permanent_uuid: "868e98cd6025463aa10e6b78dff569a7" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 35013 } } peers { permanent_uuid: "6abfe635d04e4e0d85de54a7c7efce63" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 35529 } }
I20260812 21:04:06.547399 26491 leader_election.cc:288] T cfc3265f9d4549fba44bec6968627002 P 6abfe635d04e4e0d85de54a7c7efce63 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0d68d6ebf5ac47cfa96cfe32cc083243 (127.11.254.129:36765), 868e98cd6025463aa10e6b78dff569a7 (127.11.254.130:35013)
I20260812 21:04:06.551322 26478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:06.547676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.131:47551 (local address 127.11.254.129:36765)
0812 21:04:06.547722 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0812 21:04:06.547725 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 21:04:06.547953 (+ 228us) server_negotiation.cc:407] Connection header received
0812 21:04:06.548061 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:04:06.548062 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:04:06.548071 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:04:06.548116 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:04:06.548254 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:06.548776 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:06.550398 (+ 1622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:06.550949 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:06.551032 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:06.551043 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0812 21:04:06.551194 (+ 151us) server_negotiation.cc:299] Negotiation successful
0812 21:04:06.551238 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 21:04:06.551347 26621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:06.547619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.129:36765 (local address 127.11.254.131:47551)
0812 21:04:06.547883 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0812 21:04:06.547894 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 21:04:06.547985 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:04:06.548128 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:04:06.548129 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:04:06.548131 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:04:06.548199 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:06.548200 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:06.548932 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:06.548933 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:06.550295 (+ 1362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:06.550298 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:06.551044 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:06.551045 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:06.551104 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:06.551108 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 21:04:06.551167 (+ 59us) client_negotiation.cc:240] Negotiation successful
0812 21:04:06.551212 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":225,"mutex_wait_us":66,"thread_start_us":62,"threads_started":1}
I20260812 21:04:06.552017 26476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:06.547769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.131:58335 (local address 127.11.254.130:35013)
0812 21:04:06.547812 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0812 21:04:06.547814 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:04:06.547821 (+ 7us) server_negotiation.cc:407] Connection header received
0812 21:04:06.547848 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:04:06.547849 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:04:06.547862 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:04:06.547911 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:04:06.548100 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:06.548785 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:06.551083 (+ 2298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:06.551653 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:06.551727 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:06.551735 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 21:04:06.551900 (+ 165us) server_negotiation.cc:299] Negotiation successful
0812 21:04:06.551949 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 21:04:06.552081 25911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfc3265f9d4549fba44bec6968627002" candidate_uuid: "6abfe635d04e4e0d85de54a7c7efce63" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d68d6ebf5ac47cfa96cfe32cc083243" is_pre_election: true
I20260812 21:04:06.552187 25911 raft_consensus.cc:2454] T cfc3265f9d4549fba44bec6968627002 P 0d68d6ebf5ac47cfa96cfe32cc083243 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6abfe635d04e4e0d85de54a7c7efce63 in term 0.
I20260812 21:04:06.552269 26619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:06.547584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.130:35013 (local address 127.11.254.131:58335)
0812 21:04:06.547719 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0812 21:04:06.547731 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 21:04:06.547785 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:04:06.547923 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:04:06.547924 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:04:06.547927 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:04:06.548003 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:06.548005 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:06.548883 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:06.548884 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:06.550295 (+ 1411us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:06.550298 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:06.551748 (+ 1450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:06.551749 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:06.551816 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:06.551820 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 21:04:06.552145 (+ 325us) client_negotiation.cc:240] Negotiation successful
0812 21:04:06.552191 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"mutex_wait_us":32,"thread_start_us":71,"threads_started":1}
I20260812 21:04:06.552486 26130 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfc3265f9d4549fba44bec6968627002" candidate_uuid: "6abfe635d04e4e0d85de54a7c7efce63" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "868e98cd6025463aa10e6b78dff569a7" is_pre_election: true
I20260812 21:04:06.552556 26130 raft_consensus.cc:2454] T cfc3265f9d4549fba44bec6968627002 P 868e98cd6025463aa10e6b78dff569a7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6abfe635d04e4e0d85de54a7c7efce63 in term 0.
I20260812 21:04:06.552722 26254 leader_election.cc:302] T cfc3265f9d4549fba44bec6968627002 P 6abfe635d04e4e0d85de54a7c7efce63 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6abfe635d04e4e0d85de54a7c7efce63, 868e98cd6025463aa10e6b78dff569a7; no voters:
I20260812 21:04:06.552843 26491 raft_consensus.cc:2781] T cfc3265f9d4549fba44bec6968627002 P 6abfe635d04e4e0d85de54a7c7efce63 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 21:04:06.552877 26491 raft_consensus.cc:482] T cfc3265f9d4549fba44bec6968627002 P 6abfe635d04e4e0d85de54a7c7efce63 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 21:04:06.552896 26491 raft_consensus.cc:3037] T cfc3265f9d4549fba44bec6968627002 P 6abfe635d04e4e0d85de54a7c7efce63 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:04:06.553551 26491 raft_consensus.cc:504] T cfc3265f9d4549fba44bec6968627002 P 6abfe635d04e4e0d85de54a7c7efce63 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d68d6ebf5ac47cfa96cfe32cc083243" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 36765 } } peers { permanent_uuid: "868e98cd6025463aa10e6b78dff569a7" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 35013 } } peers { permanent_uuid: "6abfe635d04e4e0d85de54a7c7efce63" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 35529 } }
I20260812 21:04:06.553696 26491 leader_election.cc:288] T cfc3265f9d4549fba44bec6968627002 P 6abfe635d04e4e0d85de54a7c7efce63 [CANDIDATE]: Term 1 election: Requested vote from peers 0d68d6ebf5ac47cfa96cfe32cc083243 (127.11.254.129:36765), 868e98cd6025463aa10e6b78dff569a7 (127.11.254.130:35013)
I20260812 21:04:06.553870 25911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfc3265f9d4549fba44bec6968627002" candidate_uuid: "6abfe635d04e4e0d85de54a7c7efce63" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d68d6ebf5ac47cfa96cfe32cc083243"
I20260812 21:04:06.553860 26130 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfc3265f9d4549fba44bec6968627002" candidate_uuid: "6abfe635d04e4e0d85de54a7c7efce63" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "868e98cd6025463aa10e6b78dff569a7"
I20260812 21:04:06.553939 26130 raft_consensus.cc:3037] T cfc3265f9d4549fba44bec6968627002 P 868e98cd6025463aa10e6b78dff569a7 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:04:06.553939 25911 raft_consensus.cc:3037] T cfc3265f9d4549fba44bec6968627002 P 0d68d6ebf5ac47cfa96cfe32cc083243 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:04:06.554728 26130 raft_consensus.cc:2454] T cfc3265f9d4549fba44bec6968627002 P 868e98cd6025463aa10e6b78dff569a7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6abfe635d04e4e0d85de54a7c7efce63 in term 1.
I20260812 21:04:06.554728 25911 raft_consensus.cc:2454] T cfc3265f9d4549fba44bec6968627002 P 0d68d6ebf5ac47cfa96cfe32cc083243 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6abfe635d04e4e0d85de54a7c7efce63 in term 1.
I20260812 21:04:06.554893 26248 leader_election.cc:302] T cfc3265f9d4549fba44bec6968627002 P 6abfe635d04e4e0d85de54a7c7efce63 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d68d6ebf5ac47cfa96cfe32cc083243, 6abfe635d04e4e0d85de54a7c7efce63; no voters:
I20260812 21:04:06.554982 26491 raft_consensus.cc:2781] T cfc3265f9d4549fba44bec6968627002 P 6abfe635d04e4e0d85de54a7c7efce63 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:04:06.555109 26491 raft_consensus.cc:686] T cfc3265f9d4549fba44bec6968627002 P 6abfe635d04e4e0d85de54a7c7efce63 [term 1 LEADER]: Becoming Leader. State: Replica: 6abfe635d04e4e0d85de54a7c7efce63, State: Running, Role: LEADER
I20260812 21:04:06.555178 26491 consensus_queue.cc:237] T cfc3265f9d4549fba44bec6968627002 P 6abfe635d04e4e0d85de54a7c7efce63 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d68d6ebf5ac47cfa96cfe32cc083243" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 36765 } } peers { permanent_uuid: "868e98cd6025463aa10e6b78dff569a7" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 35013 } } peers { permanent_uuid: "6abfe635d04e4e0d85de54a7c7efce63" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 35529 } }
I20260812 21:04:06.555954 25633 catalog_manager.cc:5495] T cfc3265f9d4549fba44bec6968627002 P 6abfe635d04e4e0d85de54a7c7efce63 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6abfe635d04e4e0d85de54a7c7efce63 (127.11.254.131). New cstate: current_term: 1 leader_uuid: "6abfe635d04e4e0d85de54a7c7efce63" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d68d6ebf5ac47cfa96cfe32cc083243" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 36765 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "868e98cd6025463aa10e6b78dff569a7" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 35013 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6abfe635d04e4e0d85de54a7c7efce63" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 35529 } health_report { overall_health: HEALTHY } } }
I20260812 21:04:06.571578 26477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:06.569087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56300 (local address 127.11.254.131:35529)
0812 21:04:06.569133 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0812 21:04:06.569135 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:04:06.569176 (+ 41us) server_negotiation.cc:407] Connection header received
0812 21:04:06.569253 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:04:06.569254 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:04:06.569276 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:04:06.569342 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 21:04:06.569465 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:06.570028 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:06.570793 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:06.571007 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:06.571060 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:06.571211 (+ 151us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 21:04:06.571359 (+ 148us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 21:04:06.571423 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0812 21:04:06.571479 (+ 56us) server_negotiation.cc:299] Negotiation successful
0812 21:04:06.571501 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 21:04:06.576122 26478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:06.574134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.131:55059 (local address 127.11.254.129:36765)
0812 21:04:06.574196 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0812 21:04:06.574198 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:04:06.574259 (+ 61us) server_negotiation.cc:407] Connection header received
0812 21:04:06.574306 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:04:06.574307 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:04:06.574316 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:04:06.574362 (+ 46us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 21:04:06.574456 (+ 94us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:06.574896 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:06.575581 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:06.575764 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:06.575809 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:06.575947 (+ 138us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 21:04:06.575971 (+ 24us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 21:04:06.576003 (+ 32us) server_negotiation.cc:1035] Waiting for connection context
0812 21:04:06.576041 (+ 38us) server_negotiation.cc:299] Negotiation successful
0812 21:04:06.576056 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260812 21:04:06.576156 26631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:06.574047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.129:36765 (local address 127.11.254.131:55059)
0812 21:04:06.574204 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0812 21:04:06.574215 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 21:04:06.574268 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:04:06.574369 (+ 101us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:04:06.574369 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:04:06.574371 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 21:04:06.574421 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:06.574422 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:06.574945 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:06.574946 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:06.575520 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:06.575522 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:06.575820 (+ 298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:06.575820 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:06.575873 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:06.575886 (+ 13us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 21:04:06.576008 (+ 122us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 21:04:06.576010 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 21:04:06.576038 (+ 28us) client_negotiation.cc:240] Negotiation successful
0812 21:04:06.576070 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":85,"threads_started":1}
I20260812 21:04:06.578168 26130 raft_consensus.cc:1261] T cfc3265f9d4549fba44bec6968627002 P 868e98cd6025463aa10e6b78dff569a7 [term 1 FOLLOWER]: Refusing update from remote peer 6abfe635d04e4e0d85de54a7c7efce63: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 21:04:06.578168 25911 raft_consensus.cc:1261] T cfc3265f9d4549fba44bec6968627002 P 0d68d6ebf5ac47cfa96cfe32cc083243 [term 1 FOLLOWER]: Refusing update from remote peer 6abfe635d04e4e0d85de54a7c7efce63: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 21:04:06.578426 26623 consensus_queue.cc:1035] T cfc3265f9d4549fba44bec6968627002 P 6abfe635d04e4e0d85de54a7c7efce63 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d68d6ebf5ac47cfa96cfe32cc083243" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 36765 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 21:04:06.578564 26491 consensus_queue.cc:1035] T cfc3265f9d4549fba44bec6968627002 P 6abfe635d04e4e0d85de54a7c7efce63 [LEADER]: Connected to new peer: Peer: permanent_uuid: "868e98cd6025463aa10e6b78dff569a7" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 35013 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 21:04:06.585048 26576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:06.583021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.131:35529 (local address 127.11.254.129:58281)
0812 21:04:06.583059 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0812 21:04:06.583070 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 21:04:06.583108 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:04:06.583246 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:04:06.583247 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:04:06.583249 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 21:04:06.583291 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:06.583292 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:06.583861 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:06.583861 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:06.584396 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:06.584398 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:06.584690 (+ 292us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:06.584691 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:06.584744 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:06.584751 (+ 7us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 21:04:06.584895 (+ 144us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 21:04:06.584896 (+ 1us) client_negotiation.cc:769] Sending connection context
0812 21:04:06.584937 (+ 41us) client_negotiation.cc:240] Negotiation successful
0812 21:04:06.584982 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19}
I20260812 21:04:06.585039 26477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:06.583105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.129:58281 (local address 127.11.254.131:35529)
0812 21:04:06.583155 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0812 21:04:06.583157 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:04:06.583164 (+ 7us) server_negotiation.cc:407] Connection header received
0812 21:04:06.583183 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:04:06.583184 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:04:06.583192 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:04:06.583240 (+ 48us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 21:04:06.583357 (+ 117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:06.583802 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:06.584459 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:06.584635 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:06.584678 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:06.584810 (+ 132us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 21:04:06.584842 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 21:04:06.584889 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0812 21:04:06.584941 (+ 52us) server_negotiation.cc:299] Negotiation successful
0812 21:04:06.584963 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 21:04:06.592401 25631 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58304:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 21:04:06.592505 25631 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58304:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 21:04:06.593101 25631 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5eb7e77e033b4ec48c7dabe9b94175eb: Sending DeleteTablet for 3 replicas of tablet cfc3265f9d4549fba44bec6968627002
I20260812 21:04:06.593329 25880 tablet_service.cc:1514] Processing DeleteTablet for tablet cfc3265f9d4549fba44bec6968627002 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:04:06 UTC) from {username='oryx', principal='oryx/127.11.254.190@KRBTEST.COM'} at 127.0.0.1:35804
I20260812 21:04:06.593389 26327 tablet_service.cc:1514] Processing DeleteTablet for tablet cfc3265f9d4549fba44bec6968627002 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:04:06 UTC) from {username='oryx', principal='oryx/127.11.254.190@KRBTEST.COM'} at 127.0.0.1:56292
I20260812 21:04:06.593441 26097 tablet_service.cc:1514] Processing DeleteTablet for tablet cfc3265f9d4549fba44bec6968627002 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:04:06 UTC) from {username='oryx', principal='oryx/127.11.254.190@KRBTEST.COM'} at 127.0.0.1:58964
I20260812 21:04:06.593474 26519 tablet_replica.cc:330] T cfc3265f9d4549fba44bec6968627002 P 6abfe635d04e4e0d85de54a7c7efce63: stopping tablet replica
I20260812 21:04:06.593526 26517 tablet_replica.cc:330] T cfc3265f9d4549fba44bec6968627002 P 868e98cd6025463aa10e6b78dff569a7: stopping tablet replica
I20260812 21:04:06.593528 26516 tablet_replica.cc:330] T cfc3265f9d4549fba44bec6968627002 P 0d68d6ebf5ac47cfa96cfe32cc083243: stopping tablet replica
I20260812 21:04:06.593541 26519 raft_consensus.cc:2229] T cfc3265f9d4549fba44bec6968627002 P 6abfe635d04e4e0d85de54a7c7efce63 [term 1 LEADER]: Raft consensus shutting down.
I20260812 21:04:06.593575 26517 raft_consensus.cc:2229] T cfc3265f9d4549fba44bec6968627002 P 868e98cd6025463aa10e6b78dff569a7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 21:04:06.593582 26516 raft_consensus.cc:2229] T cfc3265f9d4549fba44bec6968627002 P 0d68d6ebf5ac47cfa96cfe32cc083243 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 21:04:06.593605 26517 raft_consensus.cc:2258] T cfc3265f9d4549fba44bec6968627002 P 868e98cd6025463aa10e6b78dff569a7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 21:04:06.593627 26516 raft_consensus.cc:2258] T cfc3265f9d4549fba44bec6968627002 P 0d68d6ebf5ac47cfa96cfe32cc083243 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 21:04:06.593641 26519 raft_consensus.cc:2258] T cfc3265f9d4549fba44bec6968627002 P 6abfe635d04e4e0d85de54a7c7efce63 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 21:04:06.594072 26517 ts_tablet_manager.cc:1905] T cfc3265f9d4549fba44bec6968627002 P 868e98cd6025463aa10e6b78dff569a7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 21:04:06.594110 26516 ts_tablet_manager.cc:1905] T cfc3265f9d4549fba44bec6968627002 P 0d68d6ebf5ac47cfa96cfe32cc083243: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 21:04:06.595579 26517 ts_tablet_manager.cc:1918] T cfc3265f9d4549fba44bec6968627002 P 868e98cd6025463aa10e6b78dff569a7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 21:04:06.595656 26517 log.cc:1196] T cfc3265f9d4549fba44bec6968627002 P 868e98cd6025463aa10e6b78dff569a7: Deleting WAL directory at /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568607354921-12282-0/minicluster-data/ts-1/wal/wals/cfc3265f9d4549fba44bec6968627002
I20260812 21:04:06.595665 26516 ts_tablet_manager.cc:1918] T cfc3265f9d4549fba44bec6968627002 P 0d68d6ebf5ac47cfa96cfe32cc083243: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 21:04:06.595731 26516 log.cc:1196] T cfc3265f9d4549fba44bec6968627002 P 0d68d6ebf5ac47cfa96cfe32cc083243: Deleting WAL directory at /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568607354921-12282-0/minicluster-data/ts-0/wal/wals/cfc3265f9d4549fba44bec6968627002
I20260812 21:04:06.595875 26517 ts_tablet_manager.cc:1939] T cfc3265f9d4549fba44bec6968627002 P 868e98cd6025463aa10e6b78dff569a7: Deleting consensus metadata
I20260812 21:04:06.595948 26516 ts_tablet_manager.cc:1939] T cfc3265f9d4549fba44bec6968627002 P 0d68d6ebf5ac47cfa96cfe32cc083243: Deleting consensus metadata
I20260812 21:04:06.596342 25613 catalog_manager.cc:4841] TS 0d68d6ebf5ac47cfa96cfe32cc083243 (127.11.254.129:36765): tablet cfc3265f9d4549fba44bec6968627002 (table test-table [id=460706200b364e76a6dff796f266d011]) successfully deleted
I20260812 21:04:06.596393 25617 catalog_manager.cc:4841] TS 868e98cd6025463aa10e6b78dff569a7 (127.11.254.130:35013): tablet cfc3265f9d4549fba44bec6968627002 (table test-table [id=460706200b364e76a6dff796f266d011]) successfully deleted
I20260812 21:04:06.596549 26519 ts_tablet_manager.cc:1905] T cfc3265f9d4549fba44bec6968627002 P 6abfe635d04e4e0d85de54a7c7efce63: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 21:04:06.596601 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 25762
I20260812 21:04:06.598062 26519 ts_tablet_manager.cc:1918] T cfc3265f9d4549fba44bec6968627002 P 6abfe635d04e4e0d85de54a7c7efce63: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 21:04:06.598124 26519 log.cc:1196] T cfc3265f9d4549fba44bec6968627002 P 6abfe635d04e4e0d85de54a7c7efce63: Deleting WAL directory at /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786568607354921-12282-0/minicluster-data/ts-2/wal/wals/cfc3265f9d4549fba44bec6968627002
I20260812 21:04:06.598345 26519 ts_tablet_manager.cc:1939] T cfc3265f9d4549fba44bec6968627002 P 6abfe635d04e4e0d85de54a7c7efce63: Deleting consensus metadata
I20260812 21:04:06.598778 25615 catalog_manager.cc:4841] TS 6abfe635d04e4e0d85de54a7c7efce63 (127.11.254.131:35529): tablet cfc3265f9d4549fba44bec6968627002 (table test-table [id=460706200b364e76a6dff796f266d011]) successfully deleted
I20260812 21:04:06.616885 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 25998
I20260812 21:04:06.633404 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 26217
I20260812 21:04:06.647292 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 25590
2026-08-12T21:04:06Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (2825 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786568607354921-12282-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 21:04:06 dist-test-slave-vpvm krb5kdc[26642](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 21:04:06 dist-test-slave-vpvm krb5kdc[26642](info): set up 2 sockets
Aug 12 21:04:06 dist-test-slave-vpvm krb5kdc[26642](info): commencing operation
krb5kdc: starting...
W20260812 21:04:07.836263 12282 mini_kdc.cc:121] Time spent starting KDC: real 1.162s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786568607354921-12282-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786568607354921-12282-0/krb5kdc/test-user.keytab.
Aug 12 21:04:07 dist-test-slave-vpvm krb5kdc[26642](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568647, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T21:04:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T21:04:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.254.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.254.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.254.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.190.keytab.
Entry for principal kudu/127.11.254.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.190.keytab.
I20260812 21:04:07.965492 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786568607354921-12282-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786568607354921-12282-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786568607354921-12282-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.254.190:39761
--webserver_interface=127.11.254.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.190.keytab
--principal=kudu/127.11.254.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.254.148:42713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.254.190:39761
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786568607354921-12282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786568607354921-12282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786568607354921-12282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:04:08.052161 26662 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:04:08.052431 26662 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:04:08.052459 26662 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:04:08.053655 26662 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 21:04:08.053711 26662 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:04:08.053727 26662 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:04:08.053741 26662 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 21:04:08.053754 26662 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 21:04:08.055357 26662 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.254.148:42713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786568607354921-12282-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786568607354921-12282-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.254.190:39761
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.190.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.254.190:39761
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.254.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26662
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786568607354921-12282-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:04:08.055588 26662 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:04:08.055810 26662 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:04:08.061048 26678 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:04:08.061128 26662 server_base.cc:1034] running on GCE node
W20260812 21:04:08.061066 26671 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:04:08.061059 26675 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:04:08.061589 26662 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:04:08.061988 26662 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:04:08.063198 26662 hybrid_clock.cc:648] HybridClock initialized: now 1786568648063171 us; error 40 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-12T21:04:08Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1420 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 21:04:08 dist-test-slave-vpvm krb5kdc[26685](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 21:04:08 dist-test-slave-vpvm krb5kdc[26685](info): set up 2 sockets
Aug 12 21:04:08 dist-test-slave-vpvm krb5kdc[26685](info): commencing operation
krb5kdc: starting...
W20260812 21:04:09.257932 12282 mini_kdc.cc:121] Time spent starting KDC: real 1.163s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/krb5kdc/test-user.keytab.
Aug 12 21:04:09 dist-test-slave-vpvm krb5kdc[26685](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568649, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T21:04:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T21:04:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.254.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.254.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.254.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.190.keytab.
Entry for principal kudu/127.11.254.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.190.keytab.
I20260812 21:04:09.386933 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.254.190:34735
--webserver_interface=127.11.254.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.190.keytab
--principal=kudu/127.11.254.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.254.148:35659
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.254.190:34735
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:04:09.469985 26707 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:04:09.470196 26707 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:04:09.470218 26707 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:04:09.471364 26707 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 21:04:09.471400 26707 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:04:09.471413 26707 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:04:09.471426 26707 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 21:04:09.471436 26707 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 21:04:09.473158 26707 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.254.148:35659
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.254.190:34735
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.190.keytab
--principal=kudu/127.11.254.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.254.190:34735
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.254.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26707
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:04:09.473364 26707 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:04:09.473551 26707 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:04:09.478428 26716 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:04:09.478416 26717 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:04:09.478502 26719 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:04:09.478531 26707 server_base.cc:1034] running on GCE node
I20260812 21:04:09.478863 26707 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:04:09.479144 26707 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:04:09.480296 26707 hybrid_clock.cc:648] HybridClock initialized: now 1786568649480283 us; error 29 us; skew 500 ppm
Aug 12 21:04:09 dist-test-slave-vpvm krb5kdc[26685](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568649, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:04:09.487097 26707 init.cc:377] Logged in from keytab as kudu/127.11.254.190@KRBTEST.COM (short username kudu)
I20260812 21:04:09.487600 26707 webserver.cc:460] Webserver started at http://127.11.254.190:36489/ using document root <none> and password file <none>
I20260812 21:04:09.487855 26707 fs_manager.cc:359] Metadata directory not provided
I20260812 21:04:09.487901 26707 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:04:09.488026 26707 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:04:09.489135 26707 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/minicluster-data/master-0/data/instance:
uuid: "4fc775261c3944b0a7a36e790c9615a6"
format_stamp: "Formatted at 2026-08-12 21:04:09 on dist-test-slave-vpvm"
server_key: "d786f7b3ae87446d2ba9c017513ed630"
server_key_iv: "42d9690336c5ddf78b9a5b52a8cb3510"
server_key_version: "encryptionkey@0"
I20260812 21:04:09.489477 26707 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/minicluster-data/master-0/wal/instance:
uuid: "4fc775261c3944b0a7a36e790c9615a6"
format_stamp: "Formatted at 2026-08-12 21:04:09 on dist-test-slave-vpvm"
server_key: "d786f7b3ae87446d2ba9c017513ed630"
server_key_iv: "42d9690336c5ddf78b9a5b52a8cb3510"
server_key_version: "encryptionkey@0"
I20260812 21:04:09.490875 26707 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 21:04:09.491629 26734 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:04:09.491778 26707 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 21:04:09.491868 26707 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/minicluster-data/master-0/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/minicluster-data/master-0/wal
uuid: "4fc775261c3944b0a7a36e790c9615a6"
format_stamp: "Formatted at 2026-08-12 21:04:09 on dist-test-slave-vpvm"
server_key: "d786f7b3ae87446d2ba9c017513ed630"
server_key_iv: "42d9690336c5ddf78b9a5b52a8cb3510"
server_key_version: "encryptionkey@0"
I20260812 21:04:09.491947 26707 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:04:09.526239 26707 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:04:09.528497 26707 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:04:09.528646 26707 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:04:09.532737 26707 rpc_server.cc:307] RPC server started. Bound to: 127.11.254.190:34735
I20260812 21:04:09.532795 26804 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.254.190:34735 every 8 connection(s)
I20260812 21:04:09.533143 26707 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
I20260812 21:04:09.534072 26806 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:04:09.536320 26806 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4fc775261c3944b0a7a36e790c9615a6: Bootstrap starting.
I20260812 21:04:09.536852 26806 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4fc775261c3944b0a7a36e790c9615a6: Neither blocks nor log segments found. Creating new log.
I20260812 21:04:09.537082 26806 log.cc:824] T 00000000000000000000000000000000 P 4fc775261c3944b0a7a36e790c9615a6: Log is configured to *not* fsync() on all Append() calls
I20260812 21:04:09.537729 26806 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4fc775261c3944b0a7a36e790c9615a6: No bootstrap required, opened a new log
I20260812 21:04:09.539322 26806 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4fc775261c3944b0a7a36e790c9615a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4fc775261c3944b0a7a36e790c9615a6" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 34735 } }
I20260812 21:04:09.539466 26806 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4fc775261c3944b0a7a36e790c9615a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:04:09.539489 26806 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4fc775261c3944b0a7a36e790c9615a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4fc775261c3944b0a7a36e790c9615a6, State: Initialized, Role: FOLLOWER
I20260812 21:04:09.539593 26806 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4fc775261c3944b0a7a36e790c9615a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4fc775261c3944b0a7a36e790c9615a6" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 34735 } }
I20260812 21:04:09.539657 26806 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4fc775261c3944b0a7a36e790c9615a6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 21:04:09.539687 26806 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4fc775261c3944b0a7a36e790c9615a6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 21:04:09.539728 26806 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4fc775261c3944b0a7a36e790c9615a6 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:04:09.540249 26806 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4fc775261c3944b0a7a36e790c9615a6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4fc775261c3944b0a7a36e790c9615a6" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 34735 } }
I20260812 21:04:09.540350 26806 leader_election.cc:302] T 00000000000000000000000000000000 P 4fc775261c3944b0a7a36e790c9615a6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4fc775261c3944b0a7a36e790c9615a6; no voters:
I20260812 21:04:09.540510 26806 leader_election.cc:288] T 00000000000000000000000000000000 P 4fc775261c3944b0a7a36e790c9615a6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 21:04:09.540565 26809 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4fc775261c3944b0a7a36e790c9615a6 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:04:09.540777 26806 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4fc775261c3944b0a7a36e790c9615a6 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 21:04:09.540963 26809 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4fc775261c3944b0a7a36e790c9615a6 [term 1 LEADER]: Becoming Leader. State: Replica: 4fc775261c3944b0a7a36e790c9615a6, State: Running, Role: LEADER
I20260812 21:04:09.541059 26809 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4fc775261c3944b0a7a36e790c9615a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4fc775261c3944b0a7a36e790c9615a6" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 34735 } }
I20260812 21:04:09.541435 26811 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4fc775261c3944b0a7a36e790c9615a6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4fc775261c3944b0a7a36e790c9615a6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4fc775261c3944b0a7a36e790c9615a6" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 34735 } } }
I20260812 21:04:09.541450 26812 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4fc775261c3944b0a7a36e790c9615a6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4fc775261c3944b0a7a36e790c9615a6. Latest consensus state: current_term: 1 leader_uuid: "4fc775261c3944b0a7a36e790c9615a6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4fc775261c3944b0a7a36e790c9615a6" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 34735 } } }
I20260812 21:04:09.541617 26811 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4fc775261c3944b0a7a36e790c9615a6 [sys.catalog]: This master's current role is: LEADER
I20260812 21:04:09.541630 26812 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4fc775261c3944b0a7a36e790c9615a6 [sys.catalog]: This master's current role is: LEADER
I20260812 21:04:09.542088 26829 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 21:04:09.542598 26829 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 21:04:09.543036 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 26707
I20260812 21:04:09.543162 12282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/minicluster-data/master-0/wal/instance
I20260812 21:04:09.543334 12282 external_mini_cluster.cc:1428] Setting key fdacdd9984ad6e470183ea3d7b14fc1a
I20260812 21:04:09.544377 26829 catalog_manager.cc:1347] Generated new cluster ID: 11113316df2b401099cd88ec5864f889
I20260812 21:04:09.544430 26829 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 21:04:09.551652 26829 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 21:04:09.552544 26829 catalog_manager.cc:1504] Loading token signing keys...
Aug 12 21:04:09 dist-test-slave-vpvm krb5kdc[26685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568649, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.254.190@KRBTEST.COM
I20260812 21:04:09.557773 26829 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4fc775261c3944b0a7a36e790c9615a6: Generated new TSK 0
I20260812 21:04:09.558025 26829 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 21:04:09.560634 26849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:09.543801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36430 (local address 127.11.254.190:34735)
0812 21:04:09.543979 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0812 21:04:09.543988 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 21:04:09.544155 (+ 167us) server_negotiation.cc:407] Connection header received
0812 21:04:09.544334 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:04:09.550007 (+ 5673us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:04:09.550086 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:04:09.550240 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:04:09.551045 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:09.552393 (+ 1348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:09.553565 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:09.554236 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:09.554355 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:09.557376 (+ 3021us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:04:09.557382 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:04:09.557382 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:04:09.557393 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:04:09.559149 (+ 1756us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:04:09.559487 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:04:09.559490 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:04:09.559491 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:04:09.559594 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:04:09.559786 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:04:09.559787 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:04:09.559787 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:04:09.560022 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:04:09.560181 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0812 21:04:09.560286 (+ 105us) server_negotiation.cc:299] Negotiation successful
0812 21:04:09.560448 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":78,"threads_started":1}
WARNING: no policy specified for kudu/127.11.254.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.254.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.254.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.129.keytab.
Entry for principal kudu/127.11.254.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.129.keytab.
I20260812 21:04:09.586570 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.254.129:0
--local_ip_for_outbound_sockets=127.11.254.129
--webserver_interface=127.11.254.129
--webserver_port=0
--tserver_master_addrs=127.11.254.190:34735
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.129.keytab
--principal=kudu/127.11.254.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.254.148:35659
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:04:09.669585 26855 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:04:09.669808 26855 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:04:09.669833 26855 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:04:09.671066 26855 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:04:09.671115 26855 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:04:09.671144 26855 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.254.129
I20260812 21:04:09.672623 26855 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.254.148:35659
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.129.keytab
--principal=kudu/127.11.254.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.254.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.254.129
--webserver_port=0
--tserver_master_addrs=127.11.254.190:34735
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26855
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.254.129
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:04:09.672871 26855 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:04:09.673105 26855 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:04:09.678144 26865 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:04:09.678153 26861 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:04:09.678345 26855 server_base.cc:1034] running on GCE node
W20260812 21:04:09.678375 26863 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:04:09.678575 26855 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:04:09.678802 26855 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:04:09.679935 26855 hybrid_clock.cc:648] HybridClock initialized: now 1786568649679921 us; error 31 us; skew 500 ppm
Aug 12 21:04:09 dist-test-slave-vpvm krb5kdc[26685](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568649, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:04:09.682602 26855 init.cc:377] Logged in from keytab as kudu/127.11.254.129@KRBTEST.COM (short username kudu)
I20260812 21:04:09.683027 26855 webserver.cc:460] Webserver started at http://127.11.254.129:43265/ using document root <none> and password file <none>
I20260812 21:04:09.683236 26855 fs_manager.cc:359] Metadata directory not provided
I20260812 21:04:09.683284 26855 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:04:09.683410 26855 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:04:09.684459 26855 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/minicluster-data/ts-0/data/instance:
uuid: "65dcaf3a15184db2845d640d86820fbb"
format_stamp: "Formatted at 2026-08-12 21:04:09 on dist-test-slave-vpvm"
server_key: "ac63829e2e9662f120f10c486dbd9bb5"
server_key_iv: "e654e4094edb4f02acb02bbfc0877a65"
server_key_version: "encryptionkey@0"
I20260812 21:04:09.684825 26855 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/minicluster-data/ts-0/wal/instance:
uuid: "65dcaf3a15184db2845d640d86820fbb"
format_stamp: "Formatted at 2026-08-12 21:04:09 on dist-test-slave-vpvm"
server_key: "ac63829e2e9662f120f10c486dbd9bb5"
server_key_iv: "e654e4094edb4f02acb02bbfc0877a65"
server_key_version: "encryptionkey@0"
I20260812 21:04:09.686105 26855 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260812 21:04:09.686802 26876 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:04:09.686954 26855 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 21:04:09.687036 26855 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/minicluster-data/ts-0/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/minicluster-data/ts-0/wal
uuid: "65dcaf3a15184db2845d640d86820fbb"
format_stamp: "Formatted at 2026-08-12 21:04:09 on dist-test-slave-vpvm"
server_key: "ac63829e2e9662f120f10c486dbd9bb5"
server_key_iv: "e654e4094edb4f02acb02bbfc0877a65"
server_key_version: "encryptionkey@0"
I20260812 21:04:09.687103 26855 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:04:09.703109 26855 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:04:09.705291 26855 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:04:09.705438 26855 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:04:09.705667 26855 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:04:09.706020 26855 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:04:09.706058 26855 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:04:09.706089 26855 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:04:09.706112 26855 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:04:09.711654 26855 rpc_server.cc:307] RPC server started. Bound to: 127.11.254.129:39041
I20260812 21:04:09.711717 27070 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.254.129:39041 every 8 connection(s)
I20260812 21:04:09.712042 26855 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/minicluster-data/ts-0/data/info.pb
Aug 12 21:04:09 dist-test-slave-vpvm krb5kdc[26685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568649, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.129@KRBTEST.COM for kudu/127.11.254.190@KRBTEST.COM
I20260812 21:04:09.721902 26849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:09.712829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.129:47925 (local address 127.11.254.190:34735)
0812 21:04:09.712901 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0812 21:04:09.712903 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:04:09.713092 (+ 189us) server_negotiation.cc:407] Connection header received
0812 21:04:09.713586 (+ 494us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:04:09.713587 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:04:09.713597 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:04:09.713646 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:04:09.714645 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:09.715156 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:09.716477 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:09.716812 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:09.716881 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:09.719503 (+ 2622us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:04:09.719505 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:04:09.719505 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:04:09.719513 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:04:09.720714 (+ 1201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:04:09.721101 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:04:09.721102 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:04:09.721103 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:04:09.721168 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:04:09.721363 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:04:09.721363 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:04:09.721363 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:04:09.721473 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:04:09.721527 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0812 21:04:09.721748 (+ 221us) server_negotiation.cc:299] Negotiation successful
0812 21:04:09.721831 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260812 21:04:09.722102 27073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:09.712771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:34735 (local address 127.11.254.129:47925)
0812 21:04:09.712968 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0812 21:04:09.713008 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 21:04:09.713504 (+ 496us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:04:09.713711 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:04:09.713714 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:04:09.714357 (+ 643us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:04:09.714568 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:09.714574 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:09.715226 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:09.715227 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:09.716365 (+ 1138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:09.716368 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:09.716928 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:09.716929 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:09.717074 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:09.717629 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:04:09.717640 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:04:09.719352 (+ 1712us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:04:09.720790 (+ 1438us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:04:09.720797 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:04:09.720798 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:04:09.721015 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:04:09.721227 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:04:09.721227 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:04:09.721227 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:04:09.721320 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:04:09.721517 (+ 197us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:04:09.721520 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:04:09.721644 (+ 124us) client_negotiation.cc:769] Sending connection context
0812 21:04:09.721722 (+ 78us) client_negotiation.cc:240] Negotiation successful
0812 21:04:09.721848 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":81,"threads_started":1}
I20260812 21:04:09.722213 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 26855
I20260812 21:04:09.722311 12282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/minicluster-data/ts-0/wal/instance
I20260812 21:04:09.722501 12282 external_mini_cluster.cc:1428] Setting key 8649a8b404bc48db0adb26624797b19f
I20260812 21:04:09.723076 27071 heartbeater.cc:347] Connected to a master server at 127.11.254.190:34735
I20260812 21:04:09.723189 27071 heartbeater.cc:464] Registering TS with master...
I20260812 21:04:09.723397 27071 heartbeater.cc:511] Master 127.11.254.190:34735 requested a full tablet report, sending...
I20260812 21:04:09.723958 26754 ts_manager.cc:194] Registered new tserver with Master: 65dcaf3a15184db2845d640d86820fbb (127.11.254.129:39041)
I20260812 21:04:09.724790 26754 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.254.129@KRBTEST.COM'} at 127.11.254.129:47925
WARNING: no policy specified for kudu/127.11.254.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.254.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.254.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.130.keytab.
Entry for principal kudu/127.11.254.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.130.keytab.
I20260812 21:04:09.749220 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.254.130:0
--local_ip_for_outbound_sockets=127.11.254.130
--webserver_interface=127.11.254.130
--webserver_port=0
--tserver_master_addrs=127.11.254.190:34735
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.130.keytab
--principal=kudu/127.11.254.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.254.148:35659
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:04:09.832568 27081 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:04:09.832792 27081 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:04:09.832825 27081 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:04:09.834080 27081 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:04:09.834139 27081 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:04:09.834177 27081 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.254.130
I20260812 21:04:09.835707 27081 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.254.148:35659
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.130.keytab
--principal=kudu/127.11.254.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.254.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.254.130
--webserver_port=0
--tserver_master_addrs=127.11.254.190:34735
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27081
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.254.130
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:04:09.835965 27081 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:04:09.836198 27081 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:04:09.841198 27091 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:04:09.841153 27093 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:04:09.841151 27098 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:04:09.841390 27081 server_base.cc:1034] running on GCE node
I20260812 21:04:09.841675 27081 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:04:09.841913 27081 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:04:09.843092 27081 hybrid_clock.cc:648] HybridClock initialized: now 1786568649843072 us; error 37 us; skew 500 ppm
Aug 12 21:04:09 dist-test-slave-vpvm krb5kdc[26685](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568649, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:04:09.845639 27081 init.cc:377] Logged in from keytab as kudu/127.11.254.130@KRBTEST.COM (short username kudu)
I20260812 21:04:09.846100 27081 webserver.cc:460] Webserver started at http://127.11.254.130:38137/ using document root <none> and password file <none>
I20260812 21:04:09.846310 27081 fs_manager.cc:359] Metadata directory not provided
I20260812 21:04:09.846354 27081 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:04:09.846478 27081 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:04:09.847527 27081 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/minicluster-data/ts-1/data/instance:
uuid: "bf636d1a093248bd811644f1d4b89d48"
format_stamp: "Formatted at 2026-08-12 21:04:09 on dist-test-slave-vpvm"
server_key: "5bf2ed444f67eab6c116177a9e95e455"
server_key_iv: "388ed19bb31059039a8fd2fbd9b7cc2a"
server_key_version: "encryptionkey@0"
I20260812 21:04:09.847885 27081 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/minicluster-data/ts-1/wal/instance:
uuid: "bf636d1a093248bd811644f1d4b89d48"
format_stamp: "Formatted at 2026-08-12 21:04:09 on dist-test-slave-vpvm"
server_key: "5bf2ed444f67eab6c116177a9e95e455"
server_key_iv: "388ed19bb31059039a8fd2fbd9b7cc2a"
server_key_version: "encryptionkey@0"
I20260812 21:04:09.849160 27081 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 21:04:09.849874 27112 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:04:09.850067 27081 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 21:04:09.850172 27081 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/minicluster-data/ts-1/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/minicluster-data/ts-1/wal
uuid: "bf636d1a093248bd811644f1d4b89d48"
format_stamp: "Formatted at 2026-08-12 21:04:09 on dist-test-slave-vpvm"
server_key: "5bf2ed444f67eab6c116177a9e95e455"
server_key_iv: "388ed19bb31059039a8fd2fbd9b7cc2a"
server_key_version: "encryptionkey@0"
I20260812 21:04:09.850238 27081 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:04:09.889611 27081 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:04:09.891680 27081 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:04:09.891832 27081 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:04:09.892071 27081 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:04:09.892400 27081 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:04:09.892436 27081 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:04:09.892465 27081 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:04:09.892489 27081 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:04:09.898378 27081 rpc_server.cc:307] RPC server started. Bound to: 127.11.254.130:38363
I20260812 21:04:09.898427 27300 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.254.130:38363 every 8 connection(s)
I20260812 21:04:09.898795 27081 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/minicluster-data/ts-1/data/info.pb
Aug 12 21:04:09 dist-test-slave-vpvm krb5kdc[26685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568649, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.130@KRBTEST.COM for kudu/127.11.254.190@KRBTEST.COM
I20260812 21:04:09.905407 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 27081
I20260812 21:04:09.905524 12282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/minicluster-data/ts-1/wal/instance
I20260812 21:04:09.905781 12282 external_mini_cluster.cc:1428] Setting key 71d8c76e654dc09ceb3c3d50b4bfce7f
I20260812 21:04:09.908047 26849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:09.899526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.130:41247 (local address 127.11.254.190:34735)
0812 21:04:09.899589 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0812 21:04:09.899591 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:04:09.899779 (+ 188us) server_negotiation.cc:407] Connection header received
0812 21:04:09.900310 (+ 531us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:04:09.900312 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:04:09.900326 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:04:09.900382 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:04:09.901392 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:09.901903 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:09.902916 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:09.903113 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:09.903185 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:09.905258 (+ 2073us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:04:09.905260 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:04:09.905261 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:04:09.905270 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:04:09.906803 (+ 1533us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:04:09.907174 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:04:09.907175 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:04:09.907175 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:04:09.907246 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:04:09.907442 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:04:09.907443 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:04:09.907443 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:04:09.907566 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:04:09.907628 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0812 21:04:09.907860 (+ 232us) server_negotiation.cc:299] Negotiation successful
0812 21:04:09.907956 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260812 21:04:09.908128 27303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:09.899492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:34735 (local address 127.11.254.130:41247)
0812 21:04:09.899649 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0812 21:04:09.899692 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 21:04:09.900197 (+ 505us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:04:09.900477 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:04:09.900482 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:04:09.901074 (+ 592us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:04:09.901281 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:09.901286 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:09.902003 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:09.902004 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:09.902822 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:09.902824 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:09.903200 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:09.903201 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:09.903331 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:09.903780 (+ 449us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:04:09.903788 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:04:09.905140 (+ 1352us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:04:09.906875 (+ 1735us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:04:09.906881 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:04:09.906882 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:04:09.907106 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:04:09.907304 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:04:09.907305 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:04:09.907305 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:04:09.907396 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:04:09.907628 (+ 232us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:04:09.907631 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:04:09.907750 (+ 119us) client_negotiation.cc:769] Sending connection context
0812 21:04:09.907826 (+ 76us) client_negotiation.cc:240] Negotiation successful
0812 21:04:09.907951 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":64,"threads_started":1}
I20260812 21:04:09.908598 27301 heartbeater.cc:347] Connected to a master server at 127.11.254.190:34735
I20260812 21:04:09.908712 27301 heartbeater.cc:464] Registering TS with master...
I20260812 21:04:09.908890 27301 heartbeater.cc:511] Master 127.11.254.190:34735 requested a full tablet report, sending...
I20260812 21:04:09.909274 26754 ts_manager.cc:194] Registered new tserver with Master: bf636d1a093248bd811644f1d4b89d48 (127.11.254.130:38363)
I20260812 21:04:09.909770 26754 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.254.130@KRBTEST.COM'} at 127.11.254.130:41247
WARNING: no policy specified for kudu/127.11.254.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.254.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.254.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.131.keytab.
Entry for principal kudu/127.11.254.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.131.keytab.
I20260812 21:04:09.930953 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.254.131:0
--local_ip_for_outbound_sockets=127.11.254.131
--webserver_interface=127.11.254.131
--webserver_port=0
--tserver_master_addrs=127.11.254.190:34735
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.131.keytab
--principal=kudu/127.11.254.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.254.148:35659
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:04:10.014498 27307 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:04:10.014760 27307 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:04:10.014786 27307 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:04:10.016099 27307 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:04:10.016148 27307 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:04:10.016176 27307 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.254.131
I20260812 21:04:10.017693 27307 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.254.148:35659
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.131.keytab
--principal=kudu/127.11.254.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.254.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.254.131
--webserver_port=0
--tserver_master_addrs=127.11.254.190:34735
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27307
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.254.131
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:04:10.017905 27307 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:04:10.018155 27307 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:04:10.023165 27318 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:04:10.023154 27317 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:04:10.023154 27321 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:04:10.023470 27307 server_base.cc:1034] running on GCE node
I20260812 21:04:10.023751 27307 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:04:10.023986 27307 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:04:10.025138 27307 hybrid_clock.cc:648] HybridClock initialized: now 1786568650025124 us; error 29 us; skew 500 ppm
Aug 12 21:04:10 dist-test-slave-vpvm krb5kdc[26685](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568650, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:04:10.027710 27307 init.cc:377] Logged in from keytab as kudu/127.11.254.131@KRBTEST.COM (short username kudu)
I20260812 21:04:10.028175 27307 webserver.cc:460] Webserver started at http://127.11.254.131:37513/ using document root <none> and password file <none>
I20260812 21:04:10.028383 27307 fs_manager.cc:359] Metadata directory not provided
I20260812 21:04:10.028429 27307 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:04:10.028553 27307 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:04:10.029593 27307 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/minicluster-data/ts-2/data/instance:
uuid: "c1260470c49a474698a830d10166c4ac"
format_stamp: "Formatted at 2026-08-12 21:04:10 on dist-test-slave-vpvm"
server_key: "e51450e3b0d4554c0a48abd51444ba3a"
server_key_iv: "86bc22624c66a4ef9eb28f69f533456e"
server_key_version: "encryptionkey@0"
I20260812 21:04:10.029985 27307 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/minicluster-data/ts-2/wal/instance:
uuid: "c1260470c49a474698a830d10166c4ac"
format_stamp: "Formatted at 2026-08-12 21:04:10 on dist-test-slave-vpvm"
server_key: "e51450e3b0d4554c0a48abd51444ba3a"
server_key_iv: "86bc22624c66a4ef9eb28f69f533456e"
server_key_version: "encryptionkey@0"
I20260812 21:04:10.031308 27307 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 21:04:10.032035 27332 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:04:10.032197 27307 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 21:04:10.032284 27307 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/minicluster-data/ts-2/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/minicluster-data/ts-2/wal
uuid: "c1260470c49a474698a830d10166c4ac"
format_stamp: "Formatted at 2026-08-12 21:04:10 on dist-test-slave-vpvm"
server_key: "e51450e3b0d4554c0a48abd51444ba3a"
server_key_iv: "86bc22624c66a4ef9eb28f69f533456e"
server_key_version: "encryptionkey@0"
I20260812 21:04:10.032351 27307 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:04:10.046905 27307 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:04:10.048924 27307 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:04:10.049070 27307 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:04:10.049305 27307 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:04:10.049626 27307 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:04:10.049660 27307 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:04:10.049691 27307 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:04:10.049714 27307 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:04:10.055495 27307 rpc_server.cc:307] RPC server started. Bound to: 127.11.254.131:44991
I20260812 21:04:10.055558 27513 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.254.131:44991 every 8 connection(s)
I20260812 21:04:10.055888 27307 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/minicluster-data/ts-2/data/info.pb
I20260812 21:04:10.056615 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 27307
I20260812 21:04:10.056725 12282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/minicluster-data/ts-2/wal/instance
I20260812 21:04:10.056916 12282 external_mini_cluster.cc:1428] Setting key cf3e7ac99afe7f66206281ff3e6e9010
Aug 12 21:04:10 dist-test-slave-vpvm krb5kdc[26685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568650, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.131@KRBTEST.COM for kudu/127.11.254.190@KRBTEST.COM
I20260812 21:04:10.065378 26849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:10.056631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.131:43567 (local address 127.11.254.190:34735)
0812 21:04:10.056705 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0812 21:04:10.056708 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 21:04:10.056920 (+ 212us) server_negotiation.cc:407] Connection header received
0812 21:04:10.057578 (+ 658us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:04:10.057579 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:04:10.057591 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:04:10.057643 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:04:10.058761 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:10.059277 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:10.060289 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:10.060499 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:10.060555 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:10.062777 (+ 2222us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:04:10.062778 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:04:10.062779 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:04:10.062786 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:04:10.064093 (+ 1307us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:04:10.064530 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:04:10.064531 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:04:10.064531 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:04:10.064600 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:04:10.064827 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:04:10.064828 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:04:10.064828 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:04:10.064949 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:04:10.065001 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0812 21:04:10.065229 (+ 228us) server_negotiation.cc:299] Negotiation successful
0812 21:04:10.065309 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260812 21:04:10.065570 27518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:10.056573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:34735 (local address 127.11.254.131:43567)
0812 21:04:10.056779 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0812 21:04:10.056823 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 21:04:10.057451 (+ 628us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:04:10.057747 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:04:10.057751 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:04:10.058454 (+ 703us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:04:10.058675 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:10.058681 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:10.059348 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:10.059348 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:10.060219 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:10.060221 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:10.060588 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:10.060589 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:10.060733 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:10.061190 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:04:10.061198 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:04:10.062680 (+ 1482us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:04:10.064180 (+ 1500us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:04:10.064186 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:04:10.064188 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:04:10.064413 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:04:10.064676 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:04:10.064677 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:04:10.064677 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:04:10.064781 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:04:10.064994 (+ 213us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:04:10.064997 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:04:10.065126 (+ 129us) client_negotiation.cc:769] Sending connection context
0812 21:04:10.065211 (+ 85us) client_negotiation.cc:240] Negotiation successful
0812 21:04:10.065360 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":114,"threads_started":1}
I20260812 21:04:10.066082 27514 heartbeater.cc:347] Connected to a master server at 127.11.254.190:34735
I20260812 21:04:10.066192 27514 heartbeater.cc:464] Registering TS with master...
I20260812 21:04:10.066383 27514 heartbeater.cc:511] Master 127.11.254.190:34735 requested a full tablet report, sending...
I20260812 21:04:10.066758 26754 ts_manager.cc:194] Registered new tserver with Master: c1260470c49a474698a830d10166c4ac (127.11.254.131:44991)
I20260812 21:04:10.067123 26754 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.254.131@KRBTEST.COM'} at 127.11.254.131:43567
I20260812 21:04:10.068858 12282 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.11.254.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.254.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.254.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/krb5kdc/oryx_127.11.254.129.keytab.
Entry for principal oryx/127.11.254.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/krb5kdc/oryx_127.11.254.129.keytab.
WARNING: no policy specified for oryx/127.11.254.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.254.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.254.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/krb5kdc/oryx_127.11.254.130.keytab.
Entry for principal oryx/127.11.254.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/krb5kdc/oryx_127.11.254.130.keytab.
WARNING: no policy specified for oryx/127.11.254.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.254.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.254.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/krb5kdc/oryx_127.11.254.131.keytab.
Entry for principal oryx/127.11.254.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/krb5kdc/oryx_127.11.254.131.keytab.
I20260812 21:04:10.145519 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 26855
I20260812 21:04:10.158466 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 27081
I20260812 21:04:10.169823 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 27307
I20260812 21:04:10.176041 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 26707
I20260812 21:04:10.190649 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.254.190:34735
--webserver_interface=127.11.254.190
--webserver_port=36489
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.190.keytab
--principal=kudu/127.11.254.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.254.148:35659
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.254.190:34735
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:04:10.274866 27529 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:04:10.275091 27529 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:04:10.275115 27529 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:04:10.276244 27529 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 21:04:10.276279 27529 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:04:10.276292 27529 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:04:10.276304 27529 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 21:04:10.276321 27529 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 21:04:10.277756 27529 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.254.148:35659
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.254.190:34735
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.190.keytab
--principal=kudu/127.11.254.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.254.190:34735
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.254.190
--webserver_port=36489
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27529
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:04:10.277992 27529 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:04:10.278220 27529 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:04:10.283187 27539 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:04:10.283069 27536 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:04:10.283140 27529 server_base.cc:1034] running on GCE node
W20260812 21:04:10.283073 27542 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:04:10.283631 27529 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:04:10.283943 27529 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:04:10.285094 27529 hybrid_clock.cc:648] HybridClock initialized: now 1786568650285076 us; error 28 us; skew 500 ppm
Aug 12 21:04:10 dist-test-slave-vpvm krb5kdc[26685](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568650, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:04:10.287608 27529 init.cc:377] Logged in from keytab as kudu/127.11.254.190@KRBTEST.COM (short username kudu)
I20260812 21:04:10.288059 27529 webserver.cc:460] Webserver started at http://127.11.254.190:36489/ using document root <none> and password file <none>
I20260812 21:04:10.288298 27529 fs_manager.cc:359] Metadata directory not provided
I20260812 21:04:10.288365 27529 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:04:10.289708 27529 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 21:04:10.290390 27557 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:04:10.290544 27529 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 21:04:10.290632 27529 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/minicluster-data/master-0/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/minicluster-data/master-0/wal
uuid: "4fc775261c3944b0a7a36e790c9615a6"
format_stamp: "Formatted at 2026-08-12 21:04:09 on dist-test-slave-vpvm"
server_key: "d786f7b3ae87446d2ba9c017513ed630"
server_key_iv: "42d9690336c5ddf78b9a5b52a8cb3510"
server_key_version: "encryptionkey@0"
I20260812 21:04:10.290967 27529 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:04:10.304325 27529 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:04:10.306228 27529 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:04:10.306366 27529 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:04:10.310389 27529 rpc_server.cc:307] RPC server started. Bound to: 127.11.254.190:34735
I20260812 21:04:10.310426 27638 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.254.190:34735 every 8 connection(s)
I20260812 21:04:10.310789 27529 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
I20260812 21:04:10.312285 27641 sys_catalog.cc:263] Verifying existing consensus state
I20260812 21:04:10.312959 27641 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4fc775261c3944b0a7a36e790c9615a6: Bootstrap starting.
I20260812 21:04:10.315533 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 27529
I20260812 21:04:10.315789 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.254.129:39041
--local_ip_for_outbound_sockets=127.11.254.129
--tserver_master_addrs=127.11.254.190:34735
--webserver_port=43265
--webserver_interface=127.11.254.129
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.129.keytab
--principal=kudu/127.11.254.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.254.148:35659
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.11.254.129
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/krb5kdc/oryx_127.11.254.129.keytab with env {KRB5CCNAME=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 21:04:10.321220 27641 log.cc:824] T 00000000000000000000000000000000 P 4fc775261c3944b0a7a36e790c9615a6: Log is configured to *not* fsync() on all Append() calls
I20260812 21:04:10.323666 27641 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4fc775261c3944b0a7a36e790c9615a6: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 21:04:10.323982 27641 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4fc775261c3944b0a7a36e790c9615a6: Bootstrap complete.
I20260812 21:04:10.326349 27641 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4fc775261c3944b0a7a36e790c9615a6 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4fc775261c3944b0a7a36e790c9615a6" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 34735 } }
I20260812 21:04:10.326596 27641 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4fc775261c3944b0a7a36e790c9615a6 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4fc775261c3944b0a7a36e790c9615a6, State: Initialized, Role: FOLLOWER
I20260812 21:04:10.326746 27641 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4fc775261c3944b0a7a36e790c9615a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4fc775261c3944b0a7a36e790c9615a6" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 34735 } }
I20260812 21:04:10.326813 27641 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4fc775261c3944b0a7a36e790c9615a6 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 21:04:10.326844 27641 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4fc775261c3944b0a7a36e790c9615a6 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 21:04:10.326889 27641 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4fc775261c3944b0a7a36e790c9615a6 [term 1 FOLLOWER]: Advancing to term 2
I20260812 21:04:10.327960 27641 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4fc775261c3944b0a7a36e790c9615a6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4fc775261c3944b0a7a36e790c9615a6" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 34735 } }
I20260812 21:04:10.328066 27641 leader_election.cc:302] T 00000000000000000000000000000000 P 4fc775261c3944b0a7a36e790c9615a6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4fc775261c3944b0a7a36e790c9615a6; no voters:
I20260812 21:04:10.328246 27641 leader_election.cc:288] T 00000000000000000000000000000000 P 4fc775261c3944b0a7a36e790c9615a6 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260812 21:04:10.328302 27648 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4fc775261c3944b0a7a36e790c9615a6 [term 2 FOLLOWER]: Leader election won for term 2
I20260812 21:04:10.328477 27648 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4fc775261c3944b0a7a36e790c9615a6 [term 2 LEADER]: Becoming Leader. State: Replica: 4fc775261c3944b0a7a36e790c9615a6, State: Running, Role: LEADER
I20260812 21:04:10.328542 27641 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4fc775261c3944b0a7a36e790c9615a6 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 21:04:10.329044 27648 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4fc775261c3944b0a7a36e790c9615a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4fc775261c3944b0a7a36e790c9615a6" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 34735 } }
I20260812 21:04:10.329460 27650 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4fc775261c3944b0a7a36e790c9615a6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "4fc775261c3944b0a7a36e790c9615a6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4fc775261c3944b0a7a36e790c9615a6" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 34735 } } }
I20260812 21:04:10.329475 27649 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4fc775261c3944b0a7a36e790c9615a6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4fc775261c3944b0a7a36e790c9615a6. Latest consensus state: current_term: 2 leader_uuid: "4fc775261c3944b0a7a36e790c9615a6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4fc775261c3944b0a7a36e790c9615a6" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 34735 } } }
I20260812 21:04:10.329623 27650 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4fc775261c3944b0a7a36e790c9615a6 [sys.catalog]: This master's current role is: LEADER
I20260812 21:04:10.329638 27649 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4fc775261c3944b0a7a36e790c9615a6 [sys.catalog]: This master's current role is: LEADER
I20260812 21:04:10.330583 27675 catalog_manager.cc:1259] Loaded cluster ID: 11113316df2b401099cd88ec5864f889
I20260812 21:04:10.330633 27675 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 4fc775261c3944b0a7a36e790c9615a6: loading cluster ID for follower catalog manager: success
I20260812 21:04:10.331858 27675 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 4fc775261c3944b0a7a36e790c9615a6: acquiring CA information for follower catalog manager: success
I20260812 21:04:10.332237 27675 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 4fc775261c3944b0a7a36e790c9615a6: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 21:04:10.332340 27664 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 21:04:10.332531 27664 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 21:04:10.332592 27664 catalog_manager.cc:1259] Loaded cluster ID: 11113316df2b401099cd88ec5864f889
I20260812 21:04:10.332619 27664 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 21:04:10.332782 27664 catalog_manager.cc:1504] Loading token signing keys...
I20260812 21:04:10.332860 27664 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 4fc775261c3944b0a7a36e790c9615a6: Loaded TSK: 0
I20260812 21:04:10.332984 27664 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 21:04:10.402048 27643 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:04:10.402266 27643 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:04:10.402289 27643 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:04:10.403538 27643 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:04:10.403590 27643 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:04:10.403618 27643 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.254.129
I20260812 21:04:10.405182 27643 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.254.148:35659
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/krb5kdc/oryx_127.11.254.129.keytab
--principal=oryx/127.11.254.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.254.129:39041
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.254.129
--webserver_port=43265
--tserver_master_addrs=127.11.254.190:34735
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27643
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.254.129
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:04:10.405418 27643 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:04:10.405676 27643 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:04:10.411783 27685 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:04:10.411813 27687 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:04:10.411813 27683 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:04:10.412626 27643 server_base.cc:1034] running on GCE node
I20260812 21:04:10.412848 27643 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:04:10.413093 27643 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:04:10.414300 27643 hybrid_clock.cc:648] HybridClock initialized: now 1786568650414283 us; error 32 us; skew 500 ppm
Aug 12 21:04:10 dist-test-slave-vpvm krb5kdc[26685](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568650, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.254.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:04:10.416977 27643 init.cc:377] Logged in from keytab as oryx/127.11.254.129@KRBTEST.COM (short username oryx)
I20260812 21:04:10.417447 27643 webserver.cc:460] Webserver started at http://127.11.254.129:43265/ using document root <none> and password file <none>
I20260812 21:04:10.417709 27643 fs_manager.cc:359] Metadata directory not provided
I20260812 21:04:10.417784 27643 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:04:10.419230 27643 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 21:04:10.419826 27700 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:04:10.419978 27643 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 21:04:10.420058 27643 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/minicluster-data/ts-0/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/minicluster-data/ts-0/wal
uuid: "65dcaf3a15184db2845d640d86820fbb"
format_stamp: "Formatted at 2026-08-12 21:04:09 on dist-test-slave-vpvm"
server_key: "ac63829e2e9662f120f10c486dbd9bb5"
server_key_iv: "e654e4094edb4f02acb02bbfc0877a65"
server_key_version: "encryptionkey@0"
I20260812 21:04:10.420413 27643 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:04:10.436185 27643 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:04:10.438524 27643 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:04:10.438701 27643 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:04:10.438943 27643 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:04:10.439252 27643 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:04:10.439287 27643 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:04:10.439316 27643 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:04:10.439342 27643 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:04:10.445052 27643 rpc_server.cc:307] RPC server started. Bound to: 127.11.254.129:39041
I20260812 21:04:10.445122 27893 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.254.129:39041 every 8 connection(s)
I20260812 21:04:10.445487 27643 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/minicluster-data/ts-0/data/info.pb
I20260812 21:04:10.451946 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 27643
I20260812 21:04:10.452260 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.254.130:38363
--local_ip_for_outbound_sockets=127.11.254.130
--tserver_master_addrs=127.11.254.190:34735
--webserver_port=38137
--webserver_interface=127.11.254.130
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.130.keytab
--principal=kudu/127.11.254.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.254.148:35659
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.11.254.130
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/krb5kdc/oryx_127.11.254.130.keytab with env {KRB5CCNAME=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 21:04:10 dist-test-slave-vpvm krb5kdc[26685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.254.129@KRBTEST.COM for oryx/127.11.254.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 21:04:10 dist-test-slave-vpvm krb5kdc[26685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.254.129@KRBTEST.COM for oryx/127.11.254.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 21:04:10.455850 27896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:10.446258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:34735 (local address 127.11.254.129:43759)
0812 21:04:10.446450 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0812 21:04:10.446498 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 21:04:10.447112 (+ 614us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:04:10.447735 (+ 623us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:04:10.447739 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:04:10.448317 (+ 578us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:04:10.448573 (+ 256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:10.448581 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:10.450305 (+ 1724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:10.450306 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:10.451231 (+ 925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:10.451234 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:10.452192 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:10.452194 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:10.452351 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:10.452957 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:04:10.452968 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:04:10.455693 (+ 2725us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.254.190:34735: Server oryx/127.11.254.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":101,"threads_started":1}
W20260812 21:04:10.456105 27894 heartbeater.cc:644] Failed to heartbeat to 127.11.254.190:34735 (0 consecutive failures): Not authorized: Failed to ping master at 127.11.254.190:34735: Client connection negotiation failed: client connection to 127.11.254.190:34735: Server oryx/127.11.254.190@KRBTEST.COM not found in Kerberos database
W20260812 21:04:10.456538 27899 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:04:10.446380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.129:43759 (local address 127.11.254.190:34735)
0812 21:04:10.447310 (+ 930us) server_negotiation.cc:206] Beginning negotiation
0812 21:04:10.447319 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 21:04:10.447356 (+ 37us) server_negotiation.cc:407] Connection header received
0812 21:04:10.447486 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:04:10.447492 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:04:10.447544 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:04:10.447670 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:04:10.448713 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:10.450179 (+ 1466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:10.451328 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:10.452036 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:10.452192 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:10.455844 (+ 3652us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.254.129:43759: BlockingRecv error: recv got EOF from 127.11.254.129:43759 (error 108)
Metrics: {"server-negotiator.queue_time_us":877,"thread_start_us":76,"threads_started":1}
Aug 12 21:04:10 dist-test-slave-vpvm krb5kdc[26685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.254.129@KRBTEST.COM for oryx/127.11.254.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 21:04:10 dist-test-slave-vpvm krb5kdc[26685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.254.129@KRBTEST.COM for oryx/127.11.254.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 21:04:10.461638 27896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:10.456345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:34735 (local address 127.11.254.129:47513)
0812 21:04:10.456388 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0812 21:04:10.456398 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 21:04:10.456429 (+ 31us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:04:10.456702 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:04:10.456703 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:04:10.456838 (+ 135us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:04:10.456895 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:10.456897 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:10.457814 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:10.457815 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:10.458583 (+ 768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:10.458586 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:10.459050 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:10.459051 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:10.459117 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:10.459587 (+ 470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:04:10.459592 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:04:10.461571 (+ 1979us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.254.190:34735: Server oryx/127.11.254.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":28}
W20260812 21:04:10.461737 27904 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:04:10.456407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.129:47513 (local address 127.11.254.190:34735)
0812 21:04:10.456556 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0812 21:04:10.456558 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:04:10.456566 (+ 8us) server_negotiation.cc:407] Connection header received
0812 21:04:10.456590 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:04:10.456591 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:04:10.456602 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:04:10.456961 (+ 359us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:04:10.456998 (+ 37us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:10.457712 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:10.458674 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:10.458970 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:10.459045 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:10.461674 (+ 2629us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.254.129:47513: BlockingRecv error: recv got EOF from 127.11.254.129:47513 (error 108)
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":89,"threads_started":1}
Aug 12 21:04:10 dist-test-slave-vpvm krb5kdc[26685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.254.129@KRBTEST.COM for oryx/127.11.254.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 21:04:10 dist-test-slave-vpvm krb5kdc[26685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.254.129@KRBTEST.COM for oryx/127.11.254.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 21:04:10.465981 27896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:10.462054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:34735 (local address 127.11.254.129:36395)
0812 21:04:10.462097 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0812 21:04:10.462107 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 21:04:10.462143 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:04:10.462261 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:04:10.462262 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:04:10.462385 (+ 123us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:04:10.462413 (+ 28us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:10.462414 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:10.463003 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:10.463003 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:10.463497 (+ 494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:10.463499 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:10.463841 (+ 342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:10.463842 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:10.463894 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:10.464301 (+ 407us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:04:10.464304 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:04:10.465899 (+ 1595us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.254.190:34735: Server oryx/127.11.254.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":25}
W20260812 21:04:10.466060 27904 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:04:10.462125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.129:36395 (local address 127.11.254.190:34735)
0812 21:04:10.462169 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0812 21:04:10.462171 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:04:10.462179 (+ 8us) server_negotiation.cc:407] Connection header received
0812 21:04:10.462199 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:04:10.462200 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:04:10.462209 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:04:10.462253 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:04:10.462476 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:10.462938 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:10.463603 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:10.463783 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:10.463837 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:10.466013 (+ 2176us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.254.129:36395: BlockingRecv error: recv got EOF from 127.11.254.129:36395 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260812 21:04:10.466188 27894 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 21:04:10.538604 27903 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:04:10.538844 27903 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:04:10.538879 27903 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:04:10.540169 27903 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:04:10.540223 27903 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:04:10.540261 27903 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.254.130
I20260812 21:04:10.541831 27903 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.254.148:35659
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/krb5kdc/oryx_127.11.254.130.keytab
--principal=oryx/127.11.254.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.254.130:38363
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.254.130
--webserver_port=38137
--tserver_master_addrs=127.11.254.190:34735
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27903
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.254.130
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:04:10.542124 27903 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:04:10.542352 27903 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:04:10.547590 27913 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:04:10.547540 27912 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:04:10.547577 27903 server_base.cc:1034] running on GCE node
W20260812 21:04:10.547557 27915 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:04:10.548026 27903 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:04:10.548281 27903 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:04:10.549422 27903 hybrid_clock.cc:648] HybridClock initialized: now 1786568650549405 us; error 30 us; skew 500 ppm
Aug 12 21:04:10 dist-test-slave-vpvm krb5kdc[26685](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568650, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.254.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:04:10.551961 27903 init.cc:377] Logged in from keytab as oryx/127.11.254.130@KRBTEST.COM (short username oryx)
I20260812 21:04:10.552397 27903 webserver.cc:460] Webserver started at http://127.11.254.130:38137/ using document root <none> and password file <none>
I20260812 21:04:10.552587 27903 fs_manager.cc:359] Metadata directory not provided
I20260812 21:04:10.552642 27903 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:04:10.553944 27903 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 21:04:10.554656 27927 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:04:10.554829 27903 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 21:04:10.554915 27903 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/minicluster-data/ts-1/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/minicluster-data/ts-1/wal
uuid: "bf636d1a093248bd811644f1d4b89d48"
format_stamp: "Formatted at 2026-08-12 21:04:09 on dist-test-slave-vpvm"
server_key: "5bf2ed444f67eab6c116177a9e95e455"
server_key_iv: "388ed19bb31059039a8fd2fbd9b7cc2a"
server_key_version: "encryptionkey@0"
I20260812 21:04:10.555255 27903 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:04:10.572811 27903 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:04:10.574985 27903 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:04:10.575106 27903 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:04:10.575310 27903 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:04:10.575619 27903 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:04:10.575654 27903 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:04:10.575677 27903 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:04:10.575700 27903 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:04:10.581173 27903 rpc_server.cc:307] RPC server started. Bound to: 127.11.254.130:38363
I20260812 21:04:10.581250 28122 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.254.130:38363 every 8 connection(s)
I20260812 21:04:10.581563 27903 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/minicluster-data/ts-1/data/info.pb
Aug 12 21:04:10 dist-test-slave-vpvm krb5kdc[26685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.254.130@KRBTEST.COM for oryx/127.11.254.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 21:04:10 dist-test-slave-vpvm krb5kdc[26685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.254.130@KRBTEST.COM for oryx/127.11.254.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 21:04:10.588795 28128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:10.582260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:34735 (local address 127.11.254.130:38397)
0812 21:04:10.582425 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0812 21:04:10.582464 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 21:04:10.582974 (+ 510us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:04:10.583249 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:04:10.583253 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:04:10.583824 (+ 571us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:04:10.584040 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:10.584045 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:10.584706 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:10.584707 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:10.585526 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:10.585529 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:10.585863 (+ 334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:10.585864 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:10.585993 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:10.586524 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:04:10.586534 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:04:10.588634 (+ 2100us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.254.190:34735: Server oryx/127.11.254.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":89,"threads_started":1}
W20260812 21:04:10.588801 27904 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:04:10.582280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.130:38397 (local address 127.11.254.190:34735)
0812 21:04:10.582365 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0812 21:04:10.582368 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 21:04:10.582552 (+ 184us) server_negotiation.cc:407] Connection header received
0812 21:04:10.583083 (+ 531us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:04:10.583084 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:04:10.583100 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:04:10.583172 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:04:10.584127 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:10.584636 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:10.585604 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:10.585799 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:10.585854 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:10.588747 (+ 2893us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.254.130:38397: BlockingRecv error: recv got EOF from 127.11.254.130:38397 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
W20260812 21:04:10.589025 28123 heartbeater.cc:644] Failed to heartbeat to 127.11.254.190:34735 (0 consecutive failures): Not authorized: Failed to ping master at 127.11.254.190:34735: Client connection negotiation failed: client connection to 127.11.254.190:34735: Server oryx/127.11.254.190@KRBTEST.COM not found in Kerberos database
I20260812 21:04:10.589879 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 27903
I20260812 21:04:10.590174 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.254.131:44991
--local_ip_for_outbound_sockets=127.11.254.131
--tserver_master_addrs=127.11.254.190:34735
--webserver_port=37513
--webserver_interface=127.11.254.131
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.131.keytab
--principal=kudu/127.11.254.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.254.148:35659
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.11.254.131
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/krb5kdc/oryx_127.11.254.131.keytab with env {KRB5CCNAME=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 21:04:10 dist-test-slave-vpvm krb5kdc[26685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.254.130@KRBTEST.COM for oryx/127.11.254.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 21:04:10 dist-test-slave-vpvm krb5kdc[26685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.254.130@KRBTEST.COM for oryx/127.11.254.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 21:04:10.593873 28128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:10.589231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:34735 (local address 127.11.254.130:40427)
0812 21:04:10.589347 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0812 21:04:10.589356 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 21:04:10.589398 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:04:10.589509 (+ 111us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:04:10.589509 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:04:10.589635 (+ 126us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:04:10.589688 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:10.589690 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:10.590431 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:10.590432 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:10.591153 (+ 721us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:10.591156 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:10.591493 (+ 337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:10.591494 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:10.591555 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:10.591997 (+ 442us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:04:10.592001 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:04:10.593819 (+ 1818us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.254.190:34735: Server oryx/127.11.254.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":101}
W20260812 21:04:10.594022 27904 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:04:10.589297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.130:40427 (local address 127.11.254.190:34735)
0812 21:04:10.589342 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0812 21:04:10.589343 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 21:04:10.589386 (+ 43us) server_negotiation.cc:407] Connection header received
0812 21:04:10.589454 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:04:10.589455 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:04:10.589466 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:04:10.589499 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:04:10.589745 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:10.590352 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:10.591237 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:10.591428 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:10.591484 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:10.593934 (+ 2450us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.254.130:40427: BlockingRecv error: recv got EOF from 127.11.254.130:40427 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
Aug 12 21:04:10 dist-test-slave-vpvm krb5kdc[26685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.254.130@KRBTEST.COM for oryx/127.11.254.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 21:04:10 dist-test-slave-vpvm krb5kdc[26685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.254.130@KRBTEST.COM for oryx/127.11.254.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 21:04:10.598655 28128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:10.594293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:34735 (local address 127.11.254.130:44429)
0812 21:04:10.594343 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0812 21:04:10.594352 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 21:04:10.594386 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:04:10.594570 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:04:10.594571 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:04:10.594758 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:04:10.594807 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:10.594809 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:10.595515 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:10.595516 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:10.596014 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:10.596016 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:10.596357 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:10.596358 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:10.596407 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:10.596844 (+ 437us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:04:10.596848 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:04:10.598586 (+ 1738us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.254.190:34735: Server oryx/127.11.254.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":34}
W20260812 21:04:10.598755 27904 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:04:10.594364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.130:44429 (local address 127.11.254.190:34735)
0812 21:04:10.594440 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0812 21:04:10.594442 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:04:10.594449 (+ 7us) server_negotiation.cc:407] Connection header received
0812 21:04:10.594475 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:04:10.594475 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:04:10.594485 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:04:10.594535 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:04:10.594911 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:10.595438 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:10.596104 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:10.596298 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:10.596354 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:10.598702 (+ 2348us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.254.130:44429: BlockingRecv error: recv got EOF from 127.11.254.130:44429 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
W20260812 21:04:10.598793 28123 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 21:04:10.674069 28133 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:04:10.674292 28133 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:04:10.674326 28133 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:04:10.675508 28133 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:04:10.675551 28133 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:04:10.675585 28133 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.254.131
I20260812 21:04:10.677031 28133 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.254.148:35659
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/krb5kdc/oryx_127.11.254.131.keytab
--principal=oryx/127.11.254.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.254.131:44991
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.254.131
--webserver_port=37513
--tserver_master_addrs=127.11.254.190:34735
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28133
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.254.131
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:04:10.677273 28133 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:04:10.677492 28133 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:04:10.682717 28144 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:04:10.682781 28143 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:04:10.682912 28133 server_base.cc:1034] running on GCE node
W20260812 21:04:10.682717 28148 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:04:10.683195 28133 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:04:10.683423 28133 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:04:10.684571 28133 hybrid_clock.cc:648] HybridClock initialized: now 1786568650684549 us; error 32 us; skew 500 ppm
Aug 12 21:04:10 dist-test-slave-vpvm krb5kdc[26685](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568650, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.254.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:04:10.687144 28133 init.cc:377] Logged in from keytab as oryx/127.11.254.131@KRBTEST.COM (short username oryx)
I20260812 21:04:10.687588 28133 webserver.cc:460] Webserver started at http://127.11.254.131:37513/ using document root <none> and password file <none>
I20260812 21:04:10.687819 28133 fs_manager.cc:359] Metadata directory not provided
I20260812 21:04:10.687892 28133 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:04:10.689229 28133 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 21:04:10.689775 28160 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:04:10.689930 28133 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 21:04:10.690054 28133 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/minicluster-data/ts-2/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/minicluster-data/ts-2/wal
uuid: "c1260470c49a474698a830d10166c4ac"
format_stamp: "Formatted at 2026-08-12 21:04:10 on dist-test-slave-vpvm"
server_key: "e51450e3b0d4554c0a48abd51444ba3a"
server_key_iv: "86bc22624c66a4ef9eb28f69f533456e"
server_key_version: "encryptionkey@0"
I20260812 21:04:10.690413 28133 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:04:10.703733 28133 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:04:10.705551 28133 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:04:10.705698 28133 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:04:10.705931 28133 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:04:10.706300 28133 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:04:10.706346 28133 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:04:10.706375 28133 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:04:10.706400 28133 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:04:10.712106 28133 rpc_server.cc:307] RPC server started. Bound to: 127.11.254.131:44991
I20260812 21:04:10.712160 28330 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.254.131:44991 every 8 connection(s)
I20260812 21:04:10.712525 28133 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786568607354921-12282-0/minicluster-data/ts-2/data/info.pb
I20260812 21:04:10.716583 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 28133
Aug 12 21:04:10 dist-test-slave-vpvm krb5kdc[26685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.254.131@KRBTEST.COM for oryx/127.11.254.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 21:04:10 dist-test-slave-vpvm krb5kdc[26685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.254.131@KRBTEST.COM for oryx/127.11.254.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 21:04:10.720938 28334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:10.713234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:34735 (local address 127.11.254.131:38743)
0812 21:04:10.713400 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0812 21:04:10.713440 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 21:04:10.714029 (+ 589us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:04:10.714316 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:04:10.714320 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:04:10.714940 (+ 620us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:04:10.715156 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:10.715162 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:10.715856 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:10.715857 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:10.716701 (+ 844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:10.716704 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:10.717174 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:10.717176 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:10.717325 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:10.717838 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:04:10.717846 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:04:10.720707 (+ 2861us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.254.190:34735: Server oryx/127.11.254.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":86,"threads_started":1}
W20260812 21:04:10.720944 27904 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:04:10.713269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.131:38743 (local address 127.11.254.190:34735)
0812 21:04:10.713327 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0812 21:04:10.713329 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:04:10.713526 (+ 197us) server_negotiation.cc:407] Connection header received
0812 21:04:10.714159 (+ 633us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:04:10.714161 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:04:10.714175 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:04:10.714229 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:04:10.715261 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:10.715765 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:10.716811 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:10.717029 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:10.717072 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:10.720853 (+ 3781us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.254.131:38743: BlockingRecv error: recv got EOF from 127.11.254.131:38743 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
W20260812 21:04:10.721519 28332 heartbeater.cc:644] Failed to heartbeat to 127.11.254.190:34735 (0 consecutive failures): Not authorized: Failed to ping master at 127.11.254.190:34735: Client connection negotiation failed: client connection to 127.11.254.190:34735: Server oryx/127.11.254.190@KRBTEST.COM not found in Kerberos database
I20260812 21:04:10.724856 27899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:10.717183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36438 (local address 127.11.254.190:34735)
0812 21:04:10.717244 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0812 21:04:10.717246 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:04:10.717275 (+ 29us) server_negotiation.cc:407] Connection header received
0812 21:04:10.717296 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:04:10.717296 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:04:10.717311 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:04:10.717359 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:04:10.717929 (+ 570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:10.718641 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:10.719700 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:10.720009 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:10.720077 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:10.721507 (+ 1430us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:04:10.721512 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:04:10.721513 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:04:10.721526 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:04:10.723443 (+ 1917us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:04:10.723801 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:04:10.723805 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:04:10.723806 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:04:10.723937 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:04:10.724153 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:04:10.724153 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:04:10.724154 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:04:10.724400 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:04:10.724564 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0812 21:04:10.724653 (+ 89us) server_negotiation.cc:299] Negotiation successful
0812 21:04:10.724764 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
Aug 12 21:04:10 dist-test-slave-vpvm krb5kdc[26685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.254.131@KRBTEST.COM for oryx/127.11.254.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 21:04:10 dist-test-slave-vpvm krb5kdc[26685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.254.131@KRBTEST.COM for oryx/127.11.254.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 21:04:10.726652 28334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:10.721744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:34735 (local address 127.11.254.131:45489)
0812 21:04:10.721915 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0812 21:04:10.721926 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 21:04:10.721995 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:04:10.722360 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:04:10.722361 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:04:10.722546 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:04:10.722605 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:10.722607 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:10.723247 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:10.723248 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:10.723956 (+ 708us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:10.723959 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:10.724373 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:10.724374 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:10.724440 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:10.724823 (+ 383us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:04:10.724828 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:04:10.726589 (+ 1761us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.254.190:34735: Server oryx/127.11.254.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":103,"threads_started":1}
W20260812 21:04:10.726752 27904 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:04:10.722171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.131:45489 (local address 127.11.254.190:34735)
0812 21:04:10.722218 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0812 21:04:10.722220 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:04:10.722229 (+ 9us) server_negotiation.cc:407] Connection header received
0812 21:04:10.722249 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:04:10.722250 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:04:10.722259 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:04:10.722311 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:04:10.722688 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:10.723169 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:10.724032 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:10.724306 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:10.724355 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:10.726695 (+ 2340us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.254.131:45489: BlockingRecv error: recv got EOF from 127.11.254.131:45489 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
Aug 12 21:04:10 dist-test-slave-vpvm krb5kdc[26685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.254.131@KRBTEST.COM for oryx/127.11.254.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 21:04:10 dist-test-slave-vpvm krb5kdc[26685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.254.131@KRBTEST.COM for oryx/127.11.254.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 21:04:10.730998 28334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:10.726965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:34735 (local address 127.11.254.131:51113)
0812 21:04:10.727051 (+ 86us) negotiation.cc:107] Waiting for socket to connect
0812 21:04:10.727060 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 21:04:10.727102 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:04:10.727195 (+ 93us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:04:10.727195 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:04:10.727296 (+ 101us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:04:10.727324 (+ 28us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:10.727325 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:10.727917 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:10.727918 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:10.728415 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:10.728417 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:10.728815 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:10.728816 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:10.728855 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:10.729212 (+ 357us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:04:10.729216 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:04:10.730946 (+ 1730us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.254.190:34735: Server oryx/127.11.254.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":72}
W20260812 21:04:10.731087 27904 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:04:10.727013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.131:51113 (local address 127.11.254.190:34735)
0812 21:04:10.727050 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0812 21:04:10.727051 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 21:04:10.727091 (+ 40us) server_negotiation.cc:407] Connection header received
0812 21:04:10.727145 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:04:10.727145 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:04:10.727153 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:04:10.727191 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:04:10.727368 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:10.727820 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:10.728498 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:10.728733 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:10.728817 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:10.731032 (+ 2215us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.254.131:51113: BlockingRecv error: recv got EOF from 127.11.254.131:51113 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260812 21:04:10.731145 28332 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 12 21:04:11 dist-test-slave-vpvm krb5kdc[26685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.254.129@KRBTEST.COM for oryx/127.11.254.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 21:04:11 dist-test-slave-vpvm krb5kdc[26685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.254.129@KRBTEST.COM for oryx/127.11.254.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 21:04:11.471231 28339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:11.466545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:34735 (local address 127.11.254.129:60045)
0812 21:04:11.466668 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0812 21:04:11.466679 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 21:04:11.466726 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:04:11.466843 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:04:11.466844 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:04:11.467019 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:04:11.467081 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:11.467083 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:11.467785 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:11.467786 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:11.468363 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:11.468366 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:11.468758 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:11.468759 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:11.468816 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:11.469242 (+ 426us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:04:11.469246 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:04:11.471158 (+ 1912us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.254.190:34735: Server oryx/127.11.254.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":62,"threads_started":1}
W20260812 21:04:11.471344 28340 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:04:11.466587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.129:60045 (local address 127.11.254.190:34735)
0812 21:04:11.466735 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0812 21:04:11.466737 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:04:11.466744 (+ 7us) server_negotiation.cc:407] Connection header received
0812 21:04:11.466765 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:04:11.466765 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:04:11.466774 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:04:11.466833 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:04:11.467151 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:11.467703 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:11.468468 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:11.468658 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:11.468700 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:11.471270 (+ 2570us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.254.129:60045: BlockingRecv error: recv got EOF from 127.11.254.129:60045 (error 108)
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":94,"threads_started":1}
Aug 12 21:04:11 dist-test-slave-vpvm krb5kdc[26685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.254.130@KRBTEST.COM for oryx/127.11.254.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 21:04:11 dist-test-slave-vpvm krb5kdc[26685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.254.130@KRBTEST.COM for oryx/127.11.254.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 21:04:11.603891 28341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:11.599209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:34735 (local address 127.11.254.130:34091)
0812 21:04:11.599343 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0812 21:04:11.599354 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 21:04:11.599419 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:04:11.599529 (+ 110us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:04:11.599530 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:04:11.599699 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:04:11.599753 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:11.599754 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:11.600418 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:11.600419 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:11.601016 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:11.601018 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:11.601385 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:11.601386 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:11.601441 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:11.601915 (+ 474us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:04:11.601919 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:04:11.603800 (+ 1881us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.254.190:34735: Server oryx/127.11.254.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":71,"threads_started":1}
W20260812 21:04:11.603976 28340 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:04:11.599279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.130:34091 (local address 127.11.254.190:34735)
0812 21:04:11.599338 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0812 21:04:11.599340 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:04:11.599400 (+ 60us) server_negotiation.cc:407] Connection header received
0812 21:04:11.599467 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:04:11.599468 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:04:11.599478 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:04:11.599521 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:04:11.599806 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:11.600324 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:11.601083 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:11.601288 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:11.601336 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:11.603920 (+ 2584us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.254.130:34091: BlockingRecv error: recv got EOF from 127.11.254.130:34091 (error 108)
Metrics: {"server-negotiator.queue_time_us":27}
Aug 12 21:04:11 dist-test-slave-vpvm krb5kdc[26685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.254.131@KRBTEST.COM for oryx/127.11.254.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 21:04:11 dist-test-slave-vpvm krb5kdc[26685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.254.131@KRBTEST.COM for oryx/127.11.254.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 21:04:11.736225 28345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:11.731527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:34735 (local address 127.11.254.131:39105)
0812 21:04:11.731661 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0812 21:04:11.731671 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 21:04:11.731729 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:04:11.731842 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:04:11.731843 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:04:11.732020 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:04:11.732072 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:11.732074 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:11.732744 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:11.732745 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:11.733332 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:11.733334 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:11.733750 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:11.733752 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:11.733810 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:11.734314 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:04:11.734319 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:04:11.736130 (+ 1811us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.254.190:34735: Server oryx/127.11.254.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":71,"threads_started":1}
W20260812 21:04:11.736292 28340 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:04:11.731597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.131:39105 (local address 127.11.254.190:34735)
0812 21:04:11.731667 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0812 21:04:11.731669 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:04:11.731713 (+ 44us) server_negotiation.cc:407] Connection header received
0812 21:04:11.731770 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:04:11.731770 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:04:11.731781 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:04:11.731832 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:04:11.732125 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:11.732652 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:11.733400 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:11.733607 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:11.733683 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:11.736236 (+ 2553us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.254.131:39105: BlockingRecv error: recv got EOF from 127.11.254.131:39105 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 12 21:04:12 dist-test-slave-vpvm krb5kdc[26685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.254.129@KRBTEST.COM for oryx/127.11.254.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 21:04:12 dist-test-slave-vpvm krb5kdc[26685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.254.129@KRBTEST.COM for oryx/127.11.254.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 21:04:12.476984 28347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:12.471813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:34735 (local address 127.11.254.129:51111)
0812 21:04:12.471988 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0812 21:04:12.472003 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 21:04:12.472204 (+ 201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:04:12.472518 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:04:12.472520 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:04:12.472712 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:04:12.472766 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:12.472767 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:12.473523 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:12.473524 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:12.474182 (+ 658us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:12.474185 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:12.474544 (+ 359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:12.474545 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:12.474583 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:12.475084 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:04:12.475089 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:04:12.476920 (+ 1831us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.254.190:34735: Server oryx/127.11.254.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":78,"threads_started":1}
W20260812 21:04:12.477088 28348 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:04:12.471970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.129:51111 (local address 127.11.254.190:34735)
0812 21:04:12.472111 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0812 21:04:12.472113 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:04:12.472125 (+ 12us) server_negotiation.cc:407] Connection header received
0812 21:04:12.472332 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:04:12.472333 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:04:12.472350 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:04:12.472472 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:04:12.472876 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:12.473442 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:12.474272 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:12.474479 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:12.474537 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:12.477019 (+ 2482us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.254.129:51111: BlockingRecv error: recv got EOF from 127.11.254.129:51111 (error 108)
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":67,"threads_started":1}
Aug 12 21:04:12 dist-test-slave-vpvm krb5kdc[26685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.254.130@KRBTEST.COM for oryx/127.11.254.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 21:04:12 dist-test-slave-vpvm krb5kdc[26685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.254.130@KRBTEST.COM for oryx/127.11.254.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 21:04:12.609104 28349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:12.604428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:34735 (local address 127.11.254.130:49721)
0812 21:04:12.604604 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0812 21:04:12.604615 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 21:04:12.604687 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:04:12.604829 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:04:12.604829 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:04:12.605016 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:04:12.605068 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:12.605069 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:12.605736 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:12.605737 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:12.606487 (+ 750us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:12.606491 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:12.606825 (+ 334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:12.606826 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:12.606882 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:12.607296 (+ 414us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:04:12.607301 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:04:12.609042 (+ 1741us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.254.190:34735: Server oryx/127.11.254.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":98,"threads_started":1}
W20260812 21:04:12.609206 28348 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:04:12.604600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.130:49721 (local address 127.11.254.190:34735)
0812 21:04:12.604691 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0812 21:04:12.604693 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:04:12.604703 (+ 10us) server_negotiation.cc:407] Connection header received
0812 21:04:12.604749 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:04:12.604750 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:04:12.604764 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:04:12.604815 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:04:12.605144 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:12.605660 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:12.606561 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:12.606763 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:12.606819 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:12.609145 (+ 2326us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.254.130:49721: BlockingRecv error: recv got EOF from 127.11.254.130:49721 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 12 21:04:12 dist-test-slave-vpvm krb5kdc[26685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.254.131@KRBTEST.COM for oryx/127.11.254.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 21:04:12 dist-test-slave-vpvm krb5kdc[26685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.254.131@KRBTEST.COM for oryx/127.11.254.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 21:04:12.741677 28350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:12.736829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:34735 (local address 127.11.254.131:39651)
0812 21:04:12.736961 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0812 21:04:12.736974 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 21:04:12.737032 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:04:12.737206 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:04:12.737207 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:04:12.737382 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:04:12.737436 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:12.737438 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:12.738239 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:12.738241 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:12.738968 (+ 727us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:12.738972 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:12.739358 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:12.739359 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:12.739407 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:12.739861 (+ 454us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:04:12.739866 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:04:12.741612 (+ 1746us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.254.190:34735: Server oryx/127.11.254.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":65,"threads_started":1}
W20260812 21:04:12.741808 28348 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:04:12.737009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.131:39651 (local address 127.11.254.190:34735)
0812 21:04:12.737068 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0812 21:04:12.737070 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:04:12.737078 (+ 8us) server_negotiation.cc:407] Connection header received
0812 21:04:12.737110 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:04:12.737111 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:04:12.737123 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:04:12.737182 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:04:12.737537 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:12.738157 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:12.739049 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:12.739287 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:12.739360 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:12.741735 (+ 2375us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.254.131:39651: BlockingRecv error: recv got EOF from 127.11.254.131:39651 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
Aug 12 21:04:13 dist-test-slave-vpvm krb5kdc[26685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.254.129@KRBTEST.COM for oryx/127.11.254.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 21:04:13 dist-test-slave-vpvm krb5kdc[26685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.254.129@KRBTEST.COM for oryx/127.11.254.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 21:04:13.482200 28352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:13.477636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:34735 (local address 127.11.254.129:58665)
0812 21:04:13.477762 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0812 21:04:13.477774 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 21:04:13.477826 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:04:13.477996 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:04:13.477996 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:04:13.478159 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:04:13.478210 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:13.478211 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:13.478868 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:13.478869 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:13.479439 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:13.479441 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:13.479755 (+ 314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:13.479756 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:13.479794 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:13.480220 (+ 426us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:04:13.480225 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:04:13.482134 (+ 1909us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.254.190:34735: Server oryx/127.11.254.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":75,"threads_started":1}
W20260812 21:04:13.482308 28353 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:04:13.477724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.129:58665 (local address 127.11.254.190:34735)
0812 21:04:13.477857 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0812 21:04:13.477859 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:04:13.477867 (+ 8us) server_negotiation.cc:407] Connection header received
0812 21:04:13.477889 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:04:13.477889 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:04:13.477898 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:04:13.477986 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:04:13.478270 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:13.478776 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:13.479508 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:13.479702 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:13.479750 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:13.482237 (+ 2487us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.254.129:58665: BlockingRecv error: recv got EOF from 127.11.254.129:58665 (error 108)
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":65,"threads_started":1}
Aug 12 21:04:13 dist-test-slave-vpvm krb5kdc[26685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.254.130@KRBTEST.COM for oryx/127.11.254.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 21:04:13 dist-test-slave-vpvm krb5kdc[26685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.254.130@KRBTEST.COM for oryx/127.11.254.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 21:04:13.614566 28354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:13.609793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:34735 (local address 127.11.254.130:54569)
0812 21:04:13.610027 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0812 21:04:13.610041 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 21:04:13.610121 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:04:13.610251 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:04:13.610252 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:04:13.610425 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:04:13.610518 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:13.610520 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:13.611193 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:13.611195 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:13.611783 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:13.611786 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:13.612156 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:13.612157 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:13.612207 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:13.612641 (+ 434us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:04:13.612645 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:04:13.614503 (+ 1858us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.254.190:34735: Server oryx/127.11.254.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":90,"threads_started":1}
W20260812 21:04:13.614684 28353 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:04:13.609987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.130:54569 (local address 127.11.254.190:34735)
0812 21:04:13.610063 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0812 21:04:13.610065 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:04:13.610125 (+ 60us) server_negotiation.cc:407] Connection header received
0812 21:04:13.610173 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:04:13.610173 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:04:13.610185 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:04:13.610236 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:04:13.610580 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:13.611099 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:13.611873 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:13.612072 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:13.612141 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:13.614617 (+ 2476us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.254.130:54569: BlockingRecv error: recv got EOF from 127.11.254.130:54569 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 12 21:04:13 dist-test-slave-vpvm krb5kdc[26685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.254.131@KRBTEST.COM for oryx/127.11.254.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 21:04:13 dist-test-slave-vpvm krb5kdc[26685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.254.131@KRBTEST.COM for oryx/127.11.254.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 21:04:13.746723 28355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:13.742200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:34735 (local address 127.11.254.131:48043)
0812 21:04:13.742353 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0812 21:04:13.742367 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 21:04:13.742443 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:04:13.742579 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:04:13.742579 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:04:13.742760 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:04:13.742812 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:13.742814 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:13.743488 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:13.743489 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:13.744015 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:13.744017 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:13.744346 (+ 329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:13.744347 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:13.744405 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:13.744851 (+ 446us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:04:13.744856 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:04:13.746643 (+ 1787us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.254.190:34735: Server oryx/127.11.254.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":77,"threads_started":1}
W20260812 21:04:13.746801 28353 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:04:13.742288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.131:48043 (local address 127.11.254.190:34735)
0812 21:04:13.742358 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0812 21:04:13.742360 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:04:13.742427 (+ 67us) server_negotiation.cc:407] Connection header received
0812 21:04:13.742500 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:04:13.742501 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:04:13.742516 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:04:13.742566 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:04:13.742883 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:13.743409 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:13.744089 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:13.744285 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:13.744340 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:13.746746 (+ 2406us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.254.131:48043: BlockingRecv error: recv got EOF from 127.11.254.131:48043 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Aug 12 21:04:14 dist-test-slave-vpvm krb5kdc[26685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.254.129@KRBTEST.COM for oryx/127.11.254.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 21:04:14 dist-test-slave-vpvm krb5kdc[26685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.254.129@KRBTEST.COM for oryx/127.11.254.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 21:04:14.491459 28356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:14.482727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:34735 (local address 127.11.254.129:34649)
0812 21:04:14.482854 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0812 21:04:14.482866 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 21:04:14.482926 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:04:14.483048 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:04:14.483050 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:04:14.483257 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:04:14.483312 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:14.483314 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:14.483985 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:14.483986 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:14.484538 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:14.484539 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:14.484848 (+ 309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:14.484849 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:14.484887 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:14.489362 (+ 4475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:04:14.489367 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:04:14.491385 (+ 2018us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.254.190:34735: Server oryx/127.11.254.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":73,"threads_started":1}
W20260812 21:04:14.491626 28358 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:04:14.482770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.129:34649 (local address 127.11.254.190:34735)
0812 21:04:14.482921 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0812 21:04:14.482923 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:04:14.482932 (+ 9us) server_negotiation.cc:407] Connection header received
0812 21:04:14.482973 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:04:14.482973 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:04:14.482984 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:04:14.483041 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:04:14.483370 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:14.483896 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:14.484604 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:14.484797 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:14.484841 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:14.491554 (+ 6713us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.254.129:34649: BlockingRecv error: recv got EOF from 127.11.254.129:34649 (error 108)
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":64,"threads_started":1}
Aug 12 21:04:14 dist-test-slave-vpvm krb5kdc[26685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.254.130@KRBTEST.COM for oryx/127.11.254.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 21:04:14 dist-test-slave-vpvm krb5kdc[26685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.254.130@KRBTEST.COM for oryx/127.11.254.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 21:04:14.620656 28359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:14.615036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:34735 (local address 127.11.254.130:47689)
0812 21:04:14.615207 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0812 21:04:14.615218 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 21:04:14.615297 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:04:14.615490 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:04:14.615491 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:04:14.615713 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:04:14.615776 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:14.615779 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:14.616752 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:14.616754 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:14.617356 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:14.617359 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:14.617850 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:14.617851 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:14.617903 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:14.618370 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:04:14.618375 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:04:14.620562 (+ 2187us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.254.190:34735: Server oryx/127.11.254.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":95,"threads_started":1}
W20260812 21:04:14.620779 28358 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:04:14.615112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.130:47689 (local address 127.11.254.190:34735)
0812 21:04:14.615172 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0812 21:04:14.615173 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 21:04:14.615289 (+ 116us) server_negotiation.cc:407] Connection header received
0812 21:04:14.615364 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:04:14.615365 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:04:14.615380 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:04:14.615462 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:04:14.615866 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:14.616660 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:14.617454 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:14.617769 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:14.617849 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:14.620712 (+ 2863us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.254.130:47689: BlockingRecv error: recv got EOF from 127.11.254.130:47689 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
Aug 12 21:04:14 dist-test-slave-vpvm krb5kdc[26685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.254.131@KRBTEST.COM for oryx/127.11.254.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 21:04:14 dist-test-slave-vpvm krb5kdc[26685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.254.131@KRBTEST.COM for oryx/127.11.254.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 21:04:14.751966 28360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:14.747306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:34735 (local address 127.11.254.131:51149)
0812 21:04:14.747482 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0812 21:04:14.747494 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 21:04:14.747568 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:04:14.747696 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:04:14.747697 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:04:14.747862 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:04:14.747915 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:14.747917 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:14.748560 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:14.748561 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:14.749110 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:14.749113 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:14.749453 (+ 340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:14.749454 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:14.749507 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:14.749929 (+ 422us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:04:14.749933 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:04:14.751904 (+ 1971us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.254.190:34735: Server oryx/127.11.254.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":69,"threads_started":1}
W20260812 21:04:14.752120 28358 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:04:14.747421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.131:51149 (local address 127.11.254.190:34735)
0812 21:04:14.747496 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0812 21:04:14.747498 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:04:14.747570 (+ 72us) server_negotiation.cc:407] Connection header received
0812 21:04:14.747629 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:04:14.747630 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:04:14.747643 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:04:14.747682 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:04:14.747966 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:14.748472 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:14.749195 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:14.749385 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:14.749441 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:14.752046 (+ 2605us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.254.131:51149: BlockingRecv error: recv got EOF from 127.11.254.131:51149 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Aug 12 21:04:15 dist-test-slave-vpvm krb5kdc[26685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.254.129@KRBTEST.COM for oryx/127.11.254.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 21:04:15 dist-test-slave-vpvm krb5kdc[26685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.254.129@KRBTEST.COM for oryx/127.11.254.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 21:04:15.497187 28361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:15.491970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:34735 (local address 127.11.254.129:40439)
0812 21:04:15.492136 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0812 21:04:15.492149 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 21:04:15.492213 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:04:15.492350 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:04:15.492351 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:04:15.492514 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:04:15.492569 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:15.492571 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:15.493465 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:15.493466 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:15.494172 (+ 706us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:15.494176 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:15.494539 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:15.494540 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:15.494594 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:15.495080 (+ 486us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:04:15.495085 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:04:15.497124 (+ 2039us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.254.190:34735: Server oryx/127.11.254.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":71,"threads_started":1}
W20260812 21:04:15.497308 28362 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:04:15.492075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.129:40439 (local address 127.11.254.190:34735)
0812 21:04:15.492212 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0812 21:04:15.492214 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:04:15.492246 (+ 32us) server_negotiation.cc:407] Connection header received
0812 21:04:15.492273 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:04:15.492273 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:04:15.492284 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:04:15.492334 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:04:15.492718 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:15.493386 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:15.494251 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:15.494452 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:15.494506 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:15.497230 (+ 2724us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.254.129:40439: BlockingRecv error: recv got EOF from 127.11.254.129:40439 (error 108)
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":71,"threads_started":1}
Aug 12 21:04:15 dist-test-slave-vpvm krb5kdc[26685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.254.130@KRBTEST.COM for oryx/127.11.254.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 21:04:15 dist-test-slave-vpvm krb5kdc[26685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.254.130@KRBTEST.COM for oryx/127.11.254.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 21:04:15.626325 28363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:15.621409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:34735 (local address 127.11.254.130:53825)
0812 21:04:15.621559 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0812 21:04:15.621572 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 21:04:15.621641 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:04:15.621781 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:04:15.621782 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:04:15.622009 (+ 227us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:04:15.622081 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:15.622082 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:15.622931 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:15.622933 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:15.623540 (+ 607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:15.623543 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:15.623907 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:15.623907 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:15.623945 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:15.624410 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:04:15.624415 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:04:15.626244 (+ 1829us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.254.190:34735: Server oryx/127.11.254.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":68,"threads_started":1}
W20260812 21:04:15.626461 28362 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:04:15.621571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.130:53825 (local address 127.11.254.190:34735)
0812 21:04:15.621634 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0812 21:04:15.621636 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:04:15.621646 (+ 10us) server_negotiation.cc:407] Connection header received
0812 21:04:15.621707 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:04:15.621708 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:04:15.621718 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:04:15.621766 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:04:15.622139 (+ 373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:15.622852 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:15.623646 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:15.623846 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:15.623901 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:15.626351 (+ 2450us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.254.130:53825: BlockingRecv error: recv got EOF from 127.11.254.130:53825 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
Aug 12 21:04:15 dist-test-slave-vpvm krb5kdc[26685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.254.131@KRBTEST.COM for oryx/127.11.254.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 21:04:15 dist-test-slave-vpvm krb5kdc[26685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.254.131@KRBTEST.COM for oryx/127.11.254.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 21:04:15.757244 28364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:15.752449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:34735 (local address 127.11.254.131:51377)
0812 21:04:15.752580 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0812 21:04:15.752590 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 21:04:15.752681 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:04:15.752815 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:04:15.752815 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:04:15.752979 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:04:15.753031 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:15.753032 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:15.753901 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:15.753902 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:15.754480 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:15.754483 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:15.754848 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:15.754849 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:15.754893 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:15.755336 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:04:15.755341 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:04:15.757164 (+ 1823us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.254.190:34735: Server oryx/127.11.254.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":62,"threads_started":1}
W20260812 21:04:15.757350 28362 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:04:15.752559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.131:51377 (local address 127.11.254.190:34735)
0812 21:04:15.752614 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0812 21:04:15.752616 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:04:15.752686 (+ 70us) server_negotiation.cc:407] Connection header received
0812 21:04:15.752729 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:04:15.752730 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:04:15.752743 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:04:15.752801 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:04:15.753087 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:15.753831 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:15.754576 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:15.754789 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:15.754839 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:15.757293 (+ 2454us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.254.131:51377: BlockingRecv error: recv got EOF from 127.11.254.131:51377 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
Aug 12 21:04:16 dist-test-slave-vpvm krb5kdc[26685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.254.129@KRBTEST.COM for oryx/127.11.254.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 21:04:16 dist-test-slave-vpvm krb5kdc[26685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.254.129@KRBTEST.COM for oryx/127.11.254.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 21:04:16.502920 28367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:16.497750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:34735 (local address 127.11.254.129:35133)
0812 21:04:16.497899 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0812 21:04:16.497912 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 21:04:16.498009 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:04:16.498155 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:04:16.498155 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:04:16.498324 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:04:16.498377 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:16.498379 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:16.499176 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:16.499178 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:16.499893 (+ 715us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:16.499896 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:16.500264 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:16.500265 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:16.500324 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:16.500826 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:04:16.500831 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:04:16.502856 (+ 2025us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.254.190:34735: Server oryx/127.11.254.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":79,"threads_started":1}
W20260812 21:04:16.503041 28369 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:04:16.497810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.129:35133 (local address 127.11.254.190:34735)
0812 21:04:16.497984 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0812 21:04:16.497986 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:04:16.498000 (+ 14us) server_negotiation.cc:407] Connection header received
0812 21:04:16.498069 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:04:16.498070 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:04:16.498082 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:04:16.498143 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:04:16.498511 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:16.499071 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:16.499968 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:16.500177 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:16.500233 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:16.502958 (+ 2725us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.254.129:35133: BlockingRecv error: recv got EOF from 127.11.254.129:35133 (error 108)
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":74,"threads_started":1}
Aug 12 21:04:16 dist-test-slave-vpvm krb5kdc[26685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.254.130@KRBTEST.COM for oryx/127.11.254.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 21:04:16 dist-test-slave-vpvm krb5kdc[26685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.254.130@KRBTEST.COM for oryx/127.11.254.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 21:04:16.631510 28370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:16.626795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:34735 (local address 127.11.254.130:41191)
0812 21:04:16.626924 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0812 21:04:16.626934 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 21:04:16.626988 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:04:16.627132 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:04:16.627133 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:04:16.627300 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:04:16.627353 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:16.627355 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:16.627990 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:16.627991 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:16.628564 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:16.628567 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:16.628887 (+ 320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:16.628887 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:16.628924 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:16.629504 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:04:16.629509 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:04:16.631437 (+ 1928us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.254.190:34735: Server oryx/127.11.254.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":65,"threads_started":1}
W20260812 21:04:16.631635 28369 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:04:16.626904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.130:41191 (local address 127.11.254.190:34735)
0812 21:04:16.626959 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0812 21:04:16.626961 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:04:16.626972 (+ 11us) server_negotiation.cc:407] Connection header received
0812 21:04:16.627046 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:04:16.627047 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:04:16.627057 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:04:16.627119 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:04:16.627403 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:16.627915 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:16.628643 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:16.628838 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:16.628884 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:16.631572 (+ 2688us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.254.130:41191: BlockingRecv error: recv got EOF from 127.11.254.130:41191 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
Aug 12 21:04:16 dist-test-slave-vpvm krb5kdc[26685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.254.131@KRBTEST.COM for oryx/127.11.254.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 21:04:16 dist-test-slave-vpvm krb5kdc[26685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.254.131@KRBTEST.COM for oryx/127.11.254.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 21:04:16.762401 28371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:16.757835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:34735 (local address 127.11.254.131:57617)
0812 21:04:16.757991 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0812 21:04:16.758003 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 21:04:16.758064 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:04:16.758205 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:04:16.758205 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:04:16.758376 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:04:16.758429 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:16.758430 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:16.759214 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:16.759215 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:16.759777 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:16.759780 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:16.760131 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:16.760131 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:16.760173 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:16.760641 (+ 468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:04:16.760646 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:04:16.762340 (+ 1694us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.254.190:34735: Server oryx/127.11.254.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":89,"threads_started":1}
W20260812 21:04:16.762501 28369 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:04:16.757905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.131:57617 (local address 127.11.254.190:34735)
0812 21:04:16.757995 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0812 21:04:16.757997 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:04:16.758055 (+ 58us) server_negotiation.cc:407] Connection header received
0812 21:04:16.758114 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:04:16.758114 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:04:16.758136 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:04:16.758194 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:04:16.758479 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:16.759113 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:16.759857 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:16.760071 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:16.760142 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:16.762440 (+ 2298us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.254.131:57617: BlockingRecv error: recv got EOF from 127.11.254.131:57617 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
Aug 12 21:04:17 dist-test-slave-vpvm krb5kdc[26685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.254.129@KRBTEST.COM for oryx/127.11.254.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 21:04:17 dist-test-slave-vpvm krb5kdc[26685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.254.129@KRBTEST.COM for oryx/127.11.254.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 21:04:17.508227 28373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:17.503468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:34735 (local address 127.11.254.129:42195)
0812 21:04:17.503617 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0812 21:04:17.503628 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 21:04:17.503699 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:04:17.503866 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:04:17.503866 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:04:17.504037 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:04:17.504091 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:17.504093 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:17.504805 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:17.504806 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:17.505541 (+ 735us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:17.505544 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:17.505878 (+ 334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:17.505878 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:17.505913 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:17.506417 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:04:17.506422 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:04:17.508156 (+ 1734us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.254.190:34735: Server oryx/127.11.254.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":62,"threads_started":1}
W20260812 21:04:17.508327 28375 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:04:17.503604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.129:42195 (local address 127.11.254.190:34735)
0812 21:04:17.503741 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0812 21:04:17.503743 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:04:17.503753 (+ 10us) server_negotiation.cc:407] Connection header received
0812 21:04:17.503779 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:04:17.503779 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:04:17.503790 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:04:17.503850 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:04:17.504156 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:17.504702 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:17.505603 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:17.505816 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:17.505870 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:17.508256 (+ 2386us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.254.129:42195: BlockingRecv error: recv got EOF from 127.11.254.129:42195 (error 108)
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":51,"threads_started":1}
Aug 12 21:04:17 dist-test-slave-vpvm krb5kdc[26685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.254.130@KRBTEST.COM for oryx/127.11.254.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 21:04:17 dist-test-slave-vpvm krb5kdc[26685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.254.130@KRBTEST.COM for oryx/127.11.254.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 21:04:17.636865 28378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:17.632078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:34735 (local address 127.11.254.130:60025)
0812 21:04:17.632257 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0812 21:04:17.632305 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 21:04:17.632402 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:04:17.632673 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:04:17.632674 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:04:17.632850 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:04:17.632901 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:17.632903 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:17.633578 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:17.633579 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:17.634136 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:17.634138 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:17.634474 (+ 336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:17.634475 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:17.634530 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:17.634989 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:04:17.634994 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:04:17.636796 (+ 1802us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.254.190:34735: Server oryx/127.11.254.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":81,"threads_started":1}
W20260812 21:04:17.636976 28375 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:04:17.632153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.130:60025 (local address 127.11.254.190:34735)
0812 21:04:17.632224 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0812 21:04:17.632227 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 21:04:17.632407 (+ 180us) server_negotiation.cc:407] Connection header received
0812 21:04:17.632462 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:04:17.632463 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:04:17.632547 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:04:17.632638 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:04:17.632977 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:17.633484 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:17.634215 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:17.634415 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:17.634471 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:17.636904 (+ 2433us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.254.130:60025: BlockingRecv error: recv got EOF from 127.11.254.130:60025 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Aug 12 21:04:17 dist-test-slave-vpvm krb5kdc[26685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.254.131@KRBTEST.COM for oryx/127.11.254.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 21:04:17 dist-test-slave-vpvm krb5kdc[26685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.254.131@KRBTEST.COM for oryx/127.11.254.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 21:04:17.767781 28380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:17.762897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:34735 (local address 127.11.254.131:49361)
0812 21:04:17.763072 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0812 21:04:17.763086 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 21:04:17.763175 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:04:17.763309 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:04:17.763310 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:04:17.763490 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:04:17.763542 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:17.763544 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:17.764243 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:17.764244 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:17.764777 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:17.764778 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:17.765112 (+ 334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:17.765113 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:17.765169 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:17.765613 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:04:17.765617 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:04:17.767687 (+ 2070us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.254.190:34735: Server oryx/127.11.254.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":99,"threads_started":1}
W20260812 21:04:17.767855 28375 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:04:17.763000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.131:49361 (local address 127.11.254.190:34735)
0812 21:04:17.763077 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0812 21:04:17.763080 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 21:04:17.763160 (+ 80us) server_negotiation.cc:407] Connection header received
0812 21:04:17.763234 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:04:17.763235 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:04:17.763249 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:04:17.763297 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:04:17.763617 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:17.764157 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:17.764854 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:17.765049 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:17.765105 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:17.767800 (+ 2695us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.254.131:49361: BlockingRecv error: recv got EOF from 127.11.254.131:49361 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Aug 12 21:04:18 dist-test-slave-vpvm krb5kdc[26685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.254.129@KRBTEST.COM for oryx/127.11.254.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 21:04:18 dist-test-slave-vpvm krb5kdc[26685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.254.129@KRBTEST.COM for oryx/127.11.254.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 21:04:18.513761 28381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:18.508756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:34735 (local address 127.11.254.129:49733)
0812 21:04:18.508919 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0812 21:04:18.508933 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 21:04:18.508989 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:04:18.509166 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:04:18.509167 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:04:18.509391 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:04:18.509449 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:18.509452 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:18.510109 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:18.510109 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:18.510646 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:18.510647 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:18.511030 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:18.511030 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:18.511068 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:18.511613 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:04:18.511617 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:04:18.513697 (+ 2080us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.254.190:34735: Server oryx/127.11.254.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":70,"threads_started":1}
W20260812 21:04:18.513931 28383 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:04:18.508850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.129:49733 (local address 127.11.254.190:34735)
0812 21:04:18.508980 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0812 21:04:18.508983 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 21:04:18.508996 (+ 13us) server_negotiation.cc:407] Connection header received
0812 21:04:18.509045 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:04:18.509046 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:04:18.509060 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:04:18.509139 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:04:18.509510 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:18.510053 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:18.510738 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:18.510965 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:18.511028 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:18.513836 (+ 2808us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.254.129:49733: BlockingRecv error: recv got EOF from 127.11.254.129:49733 (error 108)
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":68,"threads_started":1}
Aug 12 21:04:18 dist-test-slave-vpvm krb5kdc[26685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.254.130@KRBTEST.COM for oryx/127.11.254.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 21:04:18 dist-test-slave-vpvm krb5kdc[26685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.254.130@KRBTEST.COM for oryx/127.11.254.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 21:04:18.642480 28385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:18.637427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:34735 (local address 127.11.254.130:42453)
0812 21:04:18.637610 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0812 21:04:18.637622 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 21:04:18.637696 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:04:18.637832 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:04:18.637833 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:04:18.638042 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:04:18.638094 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:18.638096 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:18.639036 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:18.639037 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:18.639662 (+ 625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:18.639665 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:18.640021 (+ 356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:18.640022 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:18.640067 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:18.640562 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:04:18.640567 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:04:18.642411 (+ 1844us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.254.190:34735: Server oryx/127.11.254.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":79,"threads_started":1}
W20260812 21:04:18.642663 28383 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:04:18.637540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.130:42453 (local address 127.11.254.190:34735)
0812 21:04:18.637597 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0812 21:04:18.637599 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:04:18.637692 (+ 93us) server_negotiation.cc:407] Connection header received
0812 21:04:18.637761 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:04:18.637763 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:04:18.637775 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:04:18.637813 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:04:18.638167 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:18.638948 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:18.639741 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:18.639954 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:18.640008 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:18.642597 (+ 2589us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.254.130:42453: BlockingRecv error: recv got EOF from 127.11.254.130:42453 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
Aug 12 21:04:18 dist-test-slave-vpvm krb5kdc[26685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.254.131@KRBTEST.COM for oryx/127.11.254.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 21:04:18 dist-test-slave-vpvm krb5kdc[26685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.254.131@KRBTEST.COM for oryx/127.11.254.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 21:04:18.773248 28386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:18.768374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:34735 (local address 127.11.254.131:43107)
0812 21:04:18.768547 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0812 21:04:18.768560 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 21:04:18.768640 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:04:18.768780 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:04:18.768781 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:04:18.768979 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:04:18.769029 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:18.769031 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:18.769944 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:18.769945 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:18.770536 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:18.770538 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:18.770892 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:18.770893 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:18.770951 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:18.771377 (+ 426us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:04:18.771382 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:04:18.773185 (+ 1803us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.254.190:34735: Server oryx/127.11.254.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":78,"threads_started":1}
W20260812 21:04:18.773361 28383 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:04:18.768459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.131:43107 (local address 127.11.254.190:34735)
0812 21:04:18.768531 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0812 21:04:18.768533 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:04:18.768629 (+ 96us) server_negotiation.cc:407] Connection header received
0812 21:04:18.768706 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:04:18.768707 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:04:18.768724 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:04:18.768776 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:04:18.769107 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:18.769861 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:18.770630 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:18.770830 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:18.770888 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:18.773288 (+ 2400us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.254.131:43107: BlockingRecv error: recv got EOF from 127.11.254.131:43107 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Aug 12 21:04:19 dist-test-slave-vpvm krb5kdc[26685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.254.129@KRBTEST.COM for oryx/127.11.254.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 21:04:19 dist-test-slave-vpvm krb5kdc[26685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.254.129@KRBTEST.COM for oryx/127.11.254.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 21:04:19.522235 28388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:19.514358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:34735 (local address 127.11.254.129:52401)
0812 21:04:19.514511 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0812 21:04:19.514522 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 21:04:19.514582 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:04:19.514763 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:04:19.514764 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:04:19.514988 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:04:19.515044 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:19.515047 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:19.515751 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:19.515752 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:19.516290 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:19.516292 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:19.516596 (+ 304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:19.516597 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:19.516640 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:19.520254 (+ 3614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:04:19.520260 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:04:19.522156 (+ 1896us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.254.190:34735: Server oryx/127.11.254.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":70,"threads_started":1}
W20260812 21:04:19.522408 28389 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:04:19.514418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.129:52401 (local address 127.11.254.190:34735)
0812 21:04:19.514557 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0812 21:04:19.514559 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:04:19.514568 (+ 9us) server_negotiation.cc:407] Connection header received
0812 21:04:19.514660 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:04:19.514661 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:04:19.514674 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:04:19.514741 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:04:19.515141 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:19.515668 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:19.516352 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:19.516545 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:19.516589 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:19.522281 (+ 5692us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.254.129:52401: BlockingRecv error: recv got EOF from 127.11.254.129:52401 (error 108)
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":82,"threads_started":1}
Aug 12 21:04:19 dist-test-slave-vpvm krb5kdc[26685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.254.130@KRBTEST.COM for oryx/127.11.254.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 21:04:19 dist-test-slave-vpvm krb5kdc[26685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.254.130@KRBTEST.COM for oryx/127.11.254.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 21:04:19.648082 28390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:19.642991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:34735 (local address 127.11.254.130:45959)
0812 21:04:19.643184 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0812 21:04:19.643198 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 21:04:19.643273 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:04:19.643432 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:04:19.643433 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:04:19.643636 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:04:19.643690 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:19.643692 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:19.644374 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:19.644375 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:19.645081 (+ 706us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:19.645085 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:19.645530 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:19.645531 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:19.645591 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:19.646092 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:04:19.646097 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:04:19.648020 (+ 1923us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.254.190:34735: Server oryx/127.11.254.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":92,"threads_started":1}
W20260812 21:04:19.648236 28389 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:04:19.643080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.130:45959 (local address 127.11.254.190:34735)
0812 21:04:19.643172 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0812 21:04:19.643175 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 21:04:19.643261 (+ 86us) server_negotiation.cc:407] Connection header received
0812 21:04:19.643345 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:04:19.643346 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:04:19.643358 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:04:19.643419 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:04:19.643769 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:19.644292 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:19.645152 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:19.645418 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:19.645483 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:19.648172 (+ 2689us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.254.130:45959: BlockingRecv error: recv got EOF from 127.11.254.130:45959 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 12 21:04:19 dist-test-slave-vpvm krb5kdc[26685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.254.131@KRBTEST.COM for oryx/127.11.254.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 21:04:19 dist-test-slave-vpvm krb5kdc[26685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.254.131@KRBTEST.COM for oryx/127.11.254.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 21:04:19.778719 28391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:19.773826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:34735 (local address 127.11.254.131:33483)
0812 21:04:19.774025 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0812 21:04:19.774039 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 21:04:19.774129 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:04:19.774280 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:04:19.774281 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:04:19.774471 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:04:19.774524 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:19.774526 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:19.775233 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:19.775234 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:19.775864 (+ 630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:19.775868 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:19.776309 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:19.776310 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:19.776371 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:19.776779 (+ 408us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:04:19.776782 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:04:19.778644 (+ 1862us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.254.190:34735: Server oryx/127.11.254.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":71,"threads_started":1}
W20260812 21:04:19.778813 28389 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:04:19.773996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.131:33483 (local address 127.11.254.190:34735)
0812 21:04:19.774046 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0812 21:04:19.774048 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:04:19.774116 (+ 68us) server_negotiation.cc:407] Connection header received
0812 21:04:19.774204 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:04:19.774205 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:04:19.774219 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:04:19.774263 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:04:19.774609 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:19.775135 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:19.775947 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:19.776241 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:19.776301 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:19.778745 (+ 2444us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.254.131:33483: BlockingRecv error: recv got EOF from 127.11.254.131:33483 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
Aug 12 21:04:20 dist-test-slave-vpvm krb5kdc[26685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.254.129@KRBTEST.COM for oryx/127.11.254.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 21:04:20 dist-test-slave-vpvm krb5kdc[26685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.254.129@KRBTEST.COM for oryx/127.11.254.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 21:04:20.527392 28392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:20.522836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:34735 (local address 127.11.254.129:35845)
0812 21:04:20.522968 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0812 21:04:20.522980 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 21:04:20.523048 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:04:20.523199 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:04:20.523200 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:04:20.523367 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:04:20.523418 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:20.523419 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:20.524144 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:20.524145 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:20.524699 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:20.524701 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:20.525010 (+ 309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:20.525011 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:20.525059 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:20.525503 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:04:20.525508 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:04:20.527309 (+ 1801us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.254.190:34735: Server oryx/127.11.254.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":69,"threads_started":1}
W20260812 21:04:20.527491 28394 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:04:20.522875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.129:35845 (local address 127.11.254.190:34735)
0812 21:04:20.523002 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0812 21:04:20.523004 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:04:20.523030 (+ 26us) server_negotiation.cc:407] Connection header received
0812 21:04:20.523109 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:04:20.523110 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:04:20.523122 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:04:20.523188 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:04:20.523482 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:20.524067 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:20.524764 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:20.524958 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:20.525006 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:20.527428 (+ 2422us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.254.129:35845: BlockingRecv error: recv got EOF from 127.11.254.129:35845 (error 108)
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":67,"threads_started":1}
Aug 12 21:04:20 dist-test-slave-vpvm krb5kdc[26685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.254.130@KRBTEST.COM for oryx/127.11.254.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 21:04:20 dist-test-slave-vpvm krb5kdc[26685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.254.130@KRBTEST.COM for oryx/127.11.254.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 21:04:20.653343 28395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:20.648678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:34735 (local address 127.11.254.130:55891)
0812 21:04:20.648807 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0812 21:04:20.648819 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 21:04:20.648889 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:04:20.649029 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:04:20.649030 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:04:20.649220 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:04:20.649270 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:20.649272 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:20.650036 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:20.650037 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:20.650588 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:20.650590 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:20.650943 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:20.650947 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:20.651003 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:20.651437 (+ 434us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:04:20.651442 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:04:20.653249 (+ 1807us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.254.190:34735: Server oryx/127.11.254.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":72,"threads_started":1}
W20260812 21:04:20.653412 28394 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:04:20.648789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.130:55891 (local address 127.11.254.190:34735)
0812 21:04:20.648845 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0812 21:04:20.648847 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:04:20.648873 (+ 26us) server_negotiation.cc:407] Connection header received
0812 21:04:20.648947 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:04:20.648948 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:04:20.648962 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:04:20.649007 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:04:20.649353 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:20.649911 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:20.650672 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:20.650867 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:20.650921 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:20.653359 (+ 2438us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.254.130:55891: BlockingRecv error: recv got EOF from 127.11.254.130:55891 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
Aug 12 21:04:20 dist-test-slave-vpvm krb5kdc[26685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.254.131@KRBTEST.COM for oryx/127.11.254.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 21:04:20 dist-test-slave-vpvm krb5kdc[26685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.254.131@KRBTEST.COM for oryx/127.11.254.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 21:04:20.783829 28397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:20.779189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:34735 (local address 127.11.254.131:42631)
0812 21:04:20.779488 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0812 21:04:20.779501 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 21:04:20.779571 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:04:20.779703 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:04:20.779704 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:04:20.779932 (+ 228us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:04:20.779997 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:20.780000 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:20.780676 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:20.780677 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:20.781223 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:20.781225 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:20.781517 (+ 292us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:20.781517 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:20.781553 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:20.781988 (+ 435us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:04:20.781992 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:04:20.783767 (+ 1775us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.254.190:34735: Server oryx/127.11.254.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":89,"threads_started":1}
W20260812 21:04:20.783929 28394 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:04:20.779372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.131:42631 (local address 127.11.254.190:34735)
0812 21:04:20.779445 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0812 21:04:20.779447 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:04:20.779573 (+ 126us) server_negotiation.cc:407] Connection header received
0812 21:04:20.779621 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:04:20.779622 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:04:20.779637 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:04:20.779679 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:04:20.780069 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:20.780576 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:20.781288 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:20.781467 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:20.781511 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:20.783865 (+ 2354us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.254.131:42631: BlockingRecv error: recv got EOF from 127.11.254.131:42631 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Aug 12 21:04:21 dist-test-slave-vpvm krb5kdc[26685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.254.129@KRBTEST.COM for oryx/127.11.254.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 21:04:21 dist-test-slave-vpvm krb5kdc[26685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.254.129@KRBTEST.COM for oryx/127.11.254.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 21:04:21.532790 28398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:21.527930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:34735 (local address 127.11.254.129:56805)
0812 21:04:21.528068 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0812 21:04:21.528080 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 21:04:21.528151 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:04:21.528342 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:04:21.528343 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:04:21.528530 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:04:21.528581 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:21.528583 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:21.529337 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:21.529338 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:21.529907 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:21.529909 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:21.530310 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:21.530312 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:21.530370 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:21.530857 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:04:21.530863 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:04:21.532696 (+ 1833us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.254.190:34735: Server oryx/127.11.254.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":77,"threads_started":1}
W20260812 21:04:21.532872 28400 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:04:21.528033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.129:56805 (local address 127.11.254.190:34735)
0812 21:04:21.528166 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0812 21:04:21.528168 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:04:21.528178 (+ 10us) server_negotiation.cc:407] Connection header received
0812 21:04:21.528205 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:04:21.528206 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:04:21.528217 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:04:21.528302 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:04:21.528702 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:21.529228 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:21.530025 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:21.530245 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:21.530304 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:21.532807 (+ 2503us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.254.129:56805: BlockingRecv error: recv got EOF from 127.11.254.129:56805 (error 108)
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":56,"threads_started":1}
Aug 12 21:04:21 dist-test-slave-vpvm krb5kdc[26685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.254.130@KRBTEST.COM for oryx/127.11.254.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 21:04:21 dist-test-slave-vpvm krb5kdc[26685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.254.130@KRBTEST.COM for oryx/127.11.254.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 21:04:21.658341 28401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:21.653898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:34735 (local address 127.11.254.130:48929)
0812 21:04:21.654091 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0812 21:04:21.654102 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 21:04:21.654185 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:04:21.654308 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:04:21.654309 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:04:21.654482 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:04:21.654532 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:21.654533 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:21.655157 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:21.655157 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:21.655701 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:21.655703 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:21.655992 (+ 289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:21.655993 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:21.656040 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:21.656485 (+ 445us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:04:21.656490 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:04:21.658248 (+ 1758us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.254.190:34735: Server oryx/127.11.254.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":121,"threads_started":1}
W20260812 21:04:21.658417 28400 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:04:21.653999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.130:48929 (local address 127.11.254.190:34735)
0812 21:04:21.654057 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0812 21:04:21.654059 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:04:21.654177 (+ 118us) server_negotiation.cc:407] Connection header received
0812 21:04:21.654231 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:04:21.654232 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:04:21.654244 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:04:21.654296 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:04:21.654579 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:21.655084 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:21.655761 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:21.655943 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:21.655986 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:21.658363 (+ 2377us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.254.130:48929: BlockingRecv error: recv got EOF from 127.11.254.130:48929 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
Aug 12 21:04:21 dist-test-slave-vpvm krb5kdc[26685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.254.131@KRBTEST.COM for oryx/127.11.254.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 21:04:21 dist-test-slave-vpvm krb5kdc[26685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.254.131@KRBTEST.COM for oryx/127.11.254.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 21:04:21.789016 28403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:21.784346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:34735 (local address 127.11.254.131:44691)
0812 21:04:21.784493 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0812 21:04:21.784505 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 21:04:21.784565 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:04:21.784675 (+ 110us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:04:21.784675 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:04:21.784853 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:04:21.784901 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:21.784903 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:21.785538 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:21.785538 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:21.786160 (+ 622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:21.786164 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:21.786517 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:21.786519 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:21.786573 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:21.787050 (+ 477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:04:21.787056 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:04:21.788942 (+ 1886us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.254.190:34735: Server oryx/127.11.254.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":70,"threads_started":1}
W20260812 21:04:21.789105 28400 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:04:21.784410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.131:44691 (local address 127.11.254.190:34735)
0812 21:04:21.784470 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0812 21:04:21.784472 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:04:21.784556 (+ 84us) server_negotiation.cc:407] Connection header received
0812 21:04:21.784607 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:04:21.784608 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:04:21.784621 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:04:21.784668 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:04:21.784952 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:21.785472 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:21.786232 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:21.786425 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:21.786481 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:21.789036 (+ 2555us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.254.131:44691: BlockingRecv error: recv got EOF from 127.11.254.131:44691 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
Aug 12 21:04:22 dist-test-slave-vpvm krb5kdc[26685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.254.129@KRBTEST.COM for oryx/127.11.254.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 21:04:22 dist-test-slave-vpvm krb5kdc[26685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.254.129@KRBTEST.COM for oryx/127.11.254.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 21:04:22.538045 28404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:22.533342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:34735 (local address 127.11.254.129:44843)
0812 21:04:22.533482 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0812 21:04:22.533494 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 21:04:22.533551 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:04:22.533731 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:04:22.533731 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:04:22.533901 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:04:22.533982 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:22.533984 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:22.534640 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:22.534640 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:22.535190 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:22.535192 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:22.535564 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:22.535565 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:22.535631 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:22.536059 (+ 428us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:04:22.536064 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:04:22.537943 (+ 1879us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.254.190:34735: Server oryx/127.11.254.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":74,"threads_started":1}
W20260812 21:04:22.538174 28405 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:04:22.533420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.129:44843 (local address 127.11.254.190:34735)
0812 21:04:22.533543 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0812 21:04:22.533545 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:04:22.533557 (+ 12us) server_negotiation.cc:407] Connection header received
0812 21:04:22.533640 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:04:22.533640 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:04:22.533652 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:04:22.533719 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:04:22.534043 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:22.534570 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:22.535262 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:22.535466 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:22.535526 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:22.538100 (+ 2574us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.254.129:44843: BlockingRecv error: recv got EOF from 127.11.254.129:44843 (error 108)
Metrics: {"server-negotiator.queue_time_us":101,"thread_start_us":61,"threads_started":1}
Aug 12 21:04:22 dist-test-slave-vpvm krb5kdc[26685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.254.130@KRBTEST.COM for oryx/127.11.254.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 21:04:22 dist-test-slave-vpvm krb5kdc[26685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.254.130@KRBTEST.COM for oryx/127.11.254.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 21:04:22.663396 28408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:22.658739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:34735 (local address 127.11.254.130:50339)
0812 21:04:22.658890 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0812 21:04:22.658912 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0812 21:04:22.658986 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:04:22.659124 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:04:22.659125 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:04:22.659313 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:04:22.659361 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:22.659363 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:22.660065 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:22.660066 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:22.660628 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:22.660630 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:22.660971 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:22.660972 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:22.661027 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:22.661508 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:04:22.661514 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:04:22.663306 (+ 1792us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.254.190:34735: Server oryx/127.11.254.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":76,"threads_started":1}
W20260812 21:04:22.663475 28405 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:04:22.658830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.130:50339 (local address 127.11.254.190:34735)
0812 21:04:22.658894 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0812 21:04:22.658896 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:04:22.658975 (+ 79us) server_negotiation.cc:407] Connection header received
0812 21:04:22.659045 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:04:22.659046 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:04:22.659058 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:04:22.659106 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:04:22.659441 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:22.659979 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:22.660717 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:22.660910 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:22.660965 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:22.663421 (+ 2456us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.254.130:50339: BlockingRecv error: recv got EOF from 127.11.254.130:50339 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Aug 12 21:04:22 dist-test-slave-vpvm krb5kdc[26685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.254.131@KRBTEST.COM for oryx/127.11.254.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 21:04:22 dist-test-slave-vpvm krb5kdc[26685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.254.131@KRBTEST.COM for oryx/127.11.254.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 21:04:22.794178 28409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:22.789505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:34735 (local address 127.11.254.131:33871)
0812 21:04:22.789652 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0812 21:04:22.789666 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 21:04:22.789743 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:04:22.789873 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:04:22.789874 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:04:22.790102 (+ 228us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:04:22.790151 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:22.790154 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:22.790803 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:22.790804 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:22.791336 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:22.791339 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:22.791695 (+ 356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:22.791696 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:22.791752 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:22.792168 (+ 416us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:04:22.792172 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:04:22.794115 (+ 1943us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.254.190:34735: Server oryx/127.11.254.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":67,"threads_started":1}
W20260812 21:04:22.794309 28405 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:04:22.789634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.131:33871 (local address 127.11.254.190:34735)
0812 21:04:22.789696 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0812 21:04:22.789698 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:04:22.789720 (+ 22us) server_negotiation.cc:407] Connection header received
0812 21:04:22.789801 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:04:22.789802 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:04:22.789815 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:04:22.789863 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:04:22.790230 (+ 367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:22.790731 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:22.791411 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:22.791625 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:22.791684 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:22.794242 (+ 2558us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.254.131:33871: BlockingRecv error: recv got EOF from 127.11.254.131:33871 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Aug 12 21:04:23 dist-test-slave-vpvm krb5kdc[26685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.254.129@KRBTEST.COM for oryx/127.11.254.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 21:04:23 dist-test-slave-vpvm krb5kdc[26685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.254.129@KRBTEST.COM for oryx/127.11.254.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 21:04:23.543258 28410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:23.538486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:34735 (local address 127.11.254.129:41147)
0812 21:04:23.538642 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0812 21:04:23.538653 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 21:04:23.538701 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:04:23.538837 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:04:23.538838 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:04:23.539017 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:04:23.539069 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:23.539071 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:23.539745 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:23.539746 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:23.540326 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:23.540329 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:23.540651 (+ 322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:23.540652 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:23.540707 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:23.541131 (+ 424us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:04:23.541135 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:04:23.543184 (+ 2049us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.254.190:34735: Server oryx/127.11.254.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":80,"threads_started":1}
W20260812 21:04:23.543368 28411 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:04:23.538539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.129:41147 (local address 127.11.254.190:34735)
0812 21:04:23.538681 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0812 21:04:23.538684 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 21:04:23.538692 (+ 8us) server_negotiation.cc:407] Connection header received
0812 21:04:23.538753 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:04:23.538754 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:04:23.538764 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:04:23.538827 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:04:23.539124 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:23.539661 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:23.540388 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:23.540575 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:23.540639 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:23.543285 (+ 2646us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.254.129:41147: BlockingRecv error: recv got EOF from 127.11.254.129:41147 (error 108)
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":57,"threads_started":1}
Aug 12 21:04:23 dist-test-slave-vpvm krb5kdc[26685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.254.130@KRBTEST.COM for oryx/127.11.254.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 21:04:23 dist-test-slave-vpvm krb5kdc[26685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.254.130@KRBTEST.COM for oryx/127.11.254.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 21:04:23.668637 28413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:23.663898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:34735 (local address 127.11.254.130:38571)
0812 21:04:23.664034 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0812 21:04:23.664046 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 21:04:23.664117 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:04:23.664277 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:04:23.664278 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:04:23.664456 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:04:23.664506 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:23.664508 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:23.665169 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:23.665170 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:23.665700 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:23.665702 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:23.666093 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:23.666094 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:23.666162 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:23.666608 (+ 446us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:04:23.666613 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:04:23.668548 (+ 1935us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.254.190:34735: Server oryx/127.11.254.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":65,"threads_started":1}
W20260812 21:04:23.668875 28411 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:04:23.664035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.130:38571 (local address 127.11.254.190:34735)
0812 21:04:23.664092 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0812 21:04:23.664095 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 21:04:23.664104 (+ 9us) server_negotiation.cc:407] Connection header received
0812 21:04:23.664202 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:04:23.664203 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:04:23.664215 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:04:23.664263 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:04:23.664578 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:23.665082 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:23.665776 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:23.666029 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:23.666091 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:23.668677 (+ 2586us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.254.130:38571: BlockingRecv error: recv got EOF from 127.11.254.130:38571 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
Aug 12 21:04:23 dist-test-slave-vpvm krb5kdc[26685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.254.131@KRBTEST.COM for oryx/127.11.254.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 21:04:23 dist-test-slave-vpvm krb5kdc[26685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.254.131@KRBTEST.COM for oryx/127.11.254.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 21:04:23.798993 28415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:23.794691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:34735 (local address 127.11.254.131:52613)
0812 21:04:23.794824 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0812 21:04:23.794835 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 21:04:23.794900 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:04:23.795026 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:04:23.795026 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:04:23.795212 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:04:23.795265 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:23.795267 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:23.795902 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:23.795903 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:23.796464 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:23.796466 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:23.796754 (+ 288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:23.796754 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:23.796792 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:23.797215 (+ 423us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:04:23.797219 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:04:23.798920 (+ 1701us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.254.190:34735: Server oryx/127.11.254.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":72,"threads_started":1}
W20260812 21:04:23.799086 28411 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:04:23.794707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.131:52613 (local address 127.11.254.190:34735)
0812 21:04:23.794767 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0812 21:04:23.794768 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 21:04:23.794892 (+ 124us) server_negotiation.cc:407] Connection header received
0812 21:04:23.794945 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:04:23.794946 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:04:23.794958 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:04:23.795014 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:04:23.795316 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:23.795847 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:23.796526 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:23.796706 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:23.796748 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:23.799022 (+ 2274us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.254.131:52613: BlockingRecv error: recv got EOF from 127.11.254.131:52613 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
Aug 12 21:04:24 dist-test-slave-vpvm krb5kdc[26685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.254.129@KRBTEST.COM for oryx/127.11.254.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 21:04:24 dist-test-slave-vpvm krb5kdc[26685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.254.129@KRBTEST.COM for oryx/127.11.254.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 21:04:24.553237 28416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:24.543785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:34735 (local address 127.11.254.129:57493)
0812 21:04:24.543939 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0812 21:04:24.543950 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 21:04:24.543999 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:04:24.544151 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:04:24.544151 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:04:24.544322 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:04:24.544374 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:24.544376 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:24.545015 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:24.545016 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:24.545558 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:24.545560 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:24.545894 (+ 334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:24.545894 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:24.545935 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:24.550982 (+ 5047us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:04:24.550989 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:04:24.553173 (+ 2184us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.254.190:34735: Server oryx/127.11.254.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":78,"threads_started":1}
W20260812 21:04:24.553390 28417 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:04:24.543888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.129:57493 (local address 127.11.254.190:34735)
0812 21:04:24.544028 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0812 21:04:24.544030 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:04:24.544039 (+ 9us) server_negotiation.cc:407] Connection header received
0812 21:04:24.544064 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:04:24.544065 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:04:24.544075 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:04:24.544144 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:04:24.544432 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:24.544956 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:24.545629 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:24.545834 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:24.545893 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:24.553307 (+ 7414us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.254.129:57493: BlockingRecv error: recv got EOF from 127.11.254.129:57493 (error 108)
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":73,"threads_started":1}
Aug 12 21:04:24 dist-test-slave-vpvm krb5kdc[26685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.254.130@KRBTEST.COM for oryx/127.11.254.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 21:04:24 dist-test-slave-vpvm krb5kdc[26685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.254.130@KRBTEST.COM for oryx/127.11.254.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 21:04:24.676033 28419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:24.671006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:34735 (local address 127.11.254.130:44623)
0812 21:04:24.671140 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0812 21:04:24.671153 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 21:04:24.671228 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:04:24.671378 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:04:24.671378 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:04:24.671603 (+ 225us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:04:24.671664 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:24.671667 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:24.672377 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:24.672378 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:24.672998 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:24.673000 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:24.673377 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:24.673378 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:24.673423 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:24.673893 (+ 470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:04:24.673899 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:04:24.675960 (+ 2061us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.254.190:34735: Server oryx/127.11.254.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":69,"threads_started":1}
W20260812 21:04:24.676180 28417 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:04:24.671051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.130:44623 (local address 127.11.254.190:34735)
0812 21:04:24.671156 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0812 21:04:24.671157 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 21:04:24.671206 (+ 49us) server_negotiation.cc:407] Connection header received
0812 21:04:24.671299 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:04:24.671299 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:04:24.671312 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:04:24.671370 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:04:24.671741 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:24.672299 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:24.673104 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:24.673312 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:24.673365 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:24.676114 (+ 2749us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.254.130:44623: BlockingRecv error: recv got EOF from 127.11.254.130:44623 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
Aug 12 21:04:24 dist-test-slave-vpvm krb5kdc[26685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.254.131@KRBTEST.COM for oryx/127.11.254.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 21:04:24 dist-test-slave-vpvm krb5kdc[26685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.254.131@KRBTEST.COM for oryx/127.11.254.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 21:04:24.804329 28422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:24.799527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:34735 (local address 127.11.254.131:40203)
0812 21:04:24.799742 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0812 21:04:24.799756 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 21:04:24.799839 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:04:24.799979 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:04:24.799979 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:04:24.800172 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:04:24.800222 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:24.800224 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:24.800923 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:24.800924 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:24.801449 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:24.801451 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:24.801776 (+ 325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:24.801777 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:24.801832 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:24.802280 (+ 448us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:04:24.802285 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:04:24.804266 (+ 1981us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.254.190:34735: Server oryx/127.11.254.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":78,"threads_started":1}
W20260812 21:04:24.804430 28417 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:04:24.799627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.131:40203 (local address 127.11.254.190:34735)
0812 21:04:24.799747 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0812 21:04:24.799750 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 21:04:24.799827 (+ 77us) server_negotiation.cc:407] Connection header received
0812 21:04:24.799894 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:04:24.799896 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:04:24.799911 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:04:24.799963 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:04:24.800294 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:24.800844 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:24.801521 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:24.801715 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:24.801771 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:24.804361 (+ 2590us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.254.131:40203: BlockingRecv error: recv got EOF from 127.11.254.131:40203 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
Aug 12 21:04:25 dist-test-slave-vpvm krb5kdc[26685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.254.129@KRBTEST.COM for oryx/127.11.254.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 21:04:25 dist-test-slave-vpvm krb5kdc[26685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.254.129@KRBTEST.COM for oryx/127.11.254.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 21:04:25.558782 28423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:25.553885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:34735 (local address 127.11.254.129:51433)
0812 21:04:25.554024 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0812 21:04:25.554036 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 21:04:25.554095 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:04:25.554298 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:04:25.554299 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:04:25.554464 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:04:25.554518 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:25.554520 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:25.555209 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:25.555210 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:25.555758 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:25.555760 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:25.556088 (+ 328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:25.556089 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:25.556132 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:25.556679 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:04:25.556686 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:04:25.558684 (+ 1998us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.254.190:34735: Server oryx/127.11.254.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":91,"threads_started":1}
W20260812 21:04:25.558864 28427 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:04:25.554035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.129:51433 (local address 127.11.254.190:34735)
0812 21:04:25.554161 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0812 21:04:25.554163 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:04:25.554172 (+ 9us) server_negotiation.cc:407] Connection header received
0812 21:04:25.554200 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:04:25.554201 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:04:25.554212 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:04:25.554286 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:04:25.554575 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:25.555117 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:25.555828 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:25.556026 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:25.556072 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:25.558799 (+ 2727us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.254.129:51433: BlockingRecv error: recv got EOF from 127.11.254.129:51433 (error 108)
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":61,"threads_started":1}
Aug 12 21:04:25 dist-test-slave-vpvm krb5kdc[26685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.254.130@KRBTEST.COM for oryx/127.11.254.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 21:04:25 dist-test-slave-vpvm krb5kdc[26685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.254.130@KRBTEST.COM for oryx/127.11.254.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 21:04:25.681522 28429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:25.676741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:34735 (local address 127.11.254.130:57515)
0812 21:04:25.676893 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0812 21:04:25.676905 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 21:04:25.676977 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:04:25.677125 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:04:25.677126 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:04:25.677363 (+ 237us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:04:25.677464 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:25.677467 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:25.678211 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:25.678212 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:25.678805 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:25.678808 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:25.679157 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:25.679158 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:25.679200 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:25.679635 (+ 435us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:04:25.679641 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:04:25.681460 (+ 1819us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.254.190:34735: Server oryx/127.11.254.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":70,"threads_started":1}
W20260812 21:04:25.681648 28427 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:04:25.676813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.130:57515 (local address 127.11.254.190:34735)
0812 21:04:25.676880 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0812 21:04:25.676882 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:04:25.676981 (+ 99us) server_negotiation.cc:407] Connection header received
0812 21:04:25.677037 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:04:25.677037 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:04:25.677051 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:04:25.677112 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:04:25.677538 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:25.678106 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:25.678890 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:25.679092 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:25.679151 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:25.681573 (+ 2422us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.254.130:57515: BlockingRecv error: recv got EOF from 127.11.254.130:57515 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
I20260812 21:04:25.719883 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 27643
I20260812 21:04:25.736588 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 27903
I20260812 21:04:25.749461 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 28133
I20260812 21:04:25.760653 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 27529
2026-08-12T21:04:25Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17699 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-12T21:04:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T21:04:25Z Disabled control of system clock
I20260812 21:04:25.803211 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786568607354921-12282-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786568607354921-12282-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786568607354921-12282-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.254.190:40569
--webserver_interface=127.11.254.190
--webserver_port=0
--builtin_ntp_servers=127.11.254.148:42629
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.254.190:40569
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260812 21:04:25.886425 28436 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:04:25.886662 28436 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:04:25.886682 28436 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:04:25.887821 28436 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 21:04:25.887856 28436 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:04:25.887867 28436 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:04:25.887878 28436 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 21:04:25.887889 28436 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 21:04:25.889232 28436 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.254.148:42629
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786568607354921-12282-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786568607354921-12282-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.254.190:40569
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.254.190:40569
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.254.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28436
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786568607354921-12282-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:04:25.889444 28436 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:04:25.889709 28436 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 21:04:25.894956 28436 server_base.cc:1034] running on GCE node
W20260812 21:04:25.894956 28443 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:04:25.894942 28442 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:04:25.895161 28446 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:04:25.895433 28436 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:04:25.895782 28436 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:04:25.896950 28436 hybrid_clock.cc:648] HybridClock initialized: now 1786568665896928 us; error 36 us; skew 500 ppm
I20260812 21:04:25.898278 28436 webserver.cc:460] Webserver started at http://127.11.254.190:39199/ using document root <none> and password file <none>
I20260812 21:04:25.898524 28436 fs_manager.cc:359] Metadata directory not provided
I20260812 21:04:25.898577 28436 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:04:25.898703 28436 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:04:25.899781 28436 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786568607354921-12282-0/minicluster-data/master-0/data/instance:
uuid: "b1b45b68856a4014be8cbe76cfffe14a"
format_stamp: "Formatted at 2026-08-12 21:04:25 on dist-test-slave-vpvm"
server_key: "1f936d19a2cb30b17a26fd4624706e5d"
server_key_iv: "530029e696e9eb4893c922a75a3041ad"
server_key_version: "encryptionkey@0"
I20260812 21:04:25.900123 28436 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786568607354921-12282-0/minicluster-data/master-0/wal/instance:
uuid: "b1b45b68856a4014be8cbe76cfffe14a"
format_stamp: "Formatted at 2026-08-12 21:04:25 on dist-test-slave-vpvm"
server_key: "1f936d19a2cb30b17a26fd4624706e5d"
server_key_iv: "530029e696e9eb4893c922a75a3041ad"
server_key_version: "encryptionkey@0"
I20260812 21:04:25.901343 28436 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 21:04:25.902065 28458 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:04:25.902232 28436 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 21:04:25.902323 28436 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786568607354921-12282-0/minicluster-data/master-0/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786568607354921-12282-0/minicluster-data/master-0/wal
uuid: "b1b45b68856a4014be8cbe76cfffe14a"
format_stamp: "Formatted at 2026-08-12 21:04:25 on dist-test-slave-vpvm"
server_key: "1f936d19a2cb30b17a26fd4624706e5d"
server_key_iv: "530029e696e9eb4893c922a75a3041ad"
server_key_version: "encryptionkey@0"
I20260812 21:04:25.902383 28436 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786568607354921-12282-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786568607354921-12282-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786568607354921-12282-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:04:25.912741 28436 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:04:25.913056 28436 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:04:25.913168 28436 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:04:25.917104 28436 rpc_server.cc:307] RPC server started. Bound to: 127.11.254.190:40569
I20260812 21:04:25.917151 28540 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.254.190:40569 every 8 connection(s)
I20260812 21:04:25.917500 28436 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
I20260812 21:04:25.917840 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 28436
I20260812 21:04:25.917924 12282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786568607354921-12282-0/minicluster-data/master-0/wal/instance
I20260812 21:04:25.918150 12282 external_mini_cluster.cc:1428] Setting key 35b9473388e11a9b500cd76c0e5a4477
I20260812 21:04:25.918325 28541 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:04:25.921605 28541 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b1b45b68856a4014be8cbe76cfffe14a: Bootstrap starting.
I20260812 21:04:25.922272 28541 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b1b45b68856a4014be8cbe76cfffe14a: Neither blocks nor log segments found. Creating new log.
I20260812 21:04:25.922529 28541 log.cc:824] T 00000000000000000000000000000000 P b1b45b68856a4014be8cbe76cfffe14a: Log is configured to *not* fsync() on all Append() calls
I20260812 21:04:25.923213 28541 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b1b45b68856a4014be8cbe76cfffe14a: No bootstrap required, opened a new log
I20260812 21:04:25.924377 28546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:25.919018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56916 (local address 127.11.254.190:40569)
0812 21:04:25.919231 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0812 21:04:25.919239 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 21:04:25.919273 (+ 34us) server_negotiation.cc:407] Connection header received
0812 21:04:25.919756 (+ 483us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:04:25.919767 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:04:25.919884 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:04:25.920089 (+ 205us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:04:25.920298 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:25.921617 (+ 1319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:25.922441 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:25.922940 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:25.923168 (+ 228us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:25.923772 (+ 604us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:04:25.923776 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:04:25.923777 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 21:04:25.923947 (+ 170us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:04:25.923980 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 21:04:25.923990 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:04:25.924045 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0812 21:04:25.924126 (+ 81us) server_negotiation.cc:299] Negotiation successful
0812 21:04:25.924213 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":70,"threads_started":1}
I20260812 21:04:25.925122 28541 raft_consensus.cc:348] T 00000000000000000000000000000000 P b1b45b68856a4014be8cbe76cfffe14a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1b45b68856a4014be8cbe76cfffe14a" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 40569 } }
I20260812 21:04:25.925262 28541 raft_consensus.cc:374] T 00000000000000000000000000000000 P b1b45b68856a4014be8cbe76cfffe14a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:04:25.925294 28541 raft_consensus.cc:729] T 00000000000000000000000000000000 P b1b45b68856a4014be8cbe76cfffe14a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1b45b68856a4014be8cbe76cfffe14a, State: Initialized, Role: FOLLOWER
I20260812 21:04:25.925410 28541 consensus_queue.cc:260] T 00000000000000000000000000000000 P b1b45b68856a4014be8cbe76cfffe14a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1b45b68856a4014be8cbe76cfffe14a" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 40569 } }
I20260812 21:04:25.925486 28541 raft_consensus.cc:388] T 00000000000000000000000000000000 P b1b45b68856a4014be8cbe76cfffe14a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 21:04:25.925525 28541 raft_consensus.cc:482] T 00000000000000000000000000000000 P b1b45b68856a4014be8cbe76cfffe14a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 21:04:25.925573 28541 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b1b45b68856a4014be8cbe76cfffe14a [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:04:25.926167 28541 raft_consensus.cc:504] T 00000000000000000000000000000000 P b1b45b68856a4014be8cbe76cfffe14a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1b45b68856a4014be8cbe76cfffe14a" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 40569 } }
I20260812 21:04:25.926275 28541 leader_election.cc:302] T 00000000000000000000000000000000 P b1b45b68856a4014be8cbe76cfffe14a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b1b45b68856a4014be8cbe76cfffe14a; no voters:
I20260812 21:04:25.926443 28541 leader_election.cc:288] T 00000000000000000000000000000000 P b1b45b68856a4014be8cbe76cfffe14a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 21:04:25.926488 28548 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b1b45b68856a4014be8cbe76cfffe14a [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:04:25.926654 28548 raft_consensus.cc:686] T 00000000000000000000000000000000 P b1b45b68856a4014be8cbe76cfffe14a [term 1 LEADER]: Becoming Leader. State: Replica: b1b45b68856a4014be8cbe76cfffe14a, State: Running, Role: LEADER
I20260812 21:04:25.926708 28541 sys_catalog.cc:564] T 00000000000000000000000000000000 P b1b45b68856a4014be8cbe76cfffe14a [sys.catalog]: configured and running, proceeding with master startup.
I20260812 21:04:25.926757 28548 consensus_queue.cc:237] T 00000000000000000000000000000000 P b1b45b68856a4014be8cbe76cfffe14a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1b45b68856a4014be8cbe76cfffe14a" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 40569 } }
I20260812 21:04:25.927155 28549 sys_catalog.cc:455] T 00000000000000000000000000000000 P b1b45b68856a4014be8cbe76cfffe14a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b1b45b68856a4014be8cbe76cfffe14a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1b45b68856a4014be8cbe76cfffe14a" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 40569 } } }
I20260812 21:04:25.927156 28556 sys_catalog.cc:455] T 00000000000000000000000000000000 P b1b45b68856a4014be8cbe76cfffe14a [sys.catalog]: SysCatalogTable state changed. Reason: New leader b1b45b68856a4014be8cbe76cfffe14a. Latest consensus state: current_term: 1 leader_uuid: "b1b45b68856a4014be8cbe76cfffe14a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1b45b68856a4014be8cbe76cfffe14a" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 40569 } } }
I20260812 21:04:25.927261 28549 sys_catalog.cc:458] T 00000000000000000000000000000000 P b1b45b68856a4014be8cbe76cfffe14a [sys.catalog]: This master's current role is: LEADER
I20260812 21:04:25.927268 28556 sys_catalog.cc:458] T 00000000000000000000000000000000 P b1b45b68856a4014be8cbe76cfffe14a [sys.catalog]: This master's current role is: LEADER
W20260812 21:04:25.928479 28578 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b1b45b68856a4014be8cbe76cfffe14a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 21:04:25.928526 28578 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 21:04:25.928593 28567 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 21:04:25.928825 28567 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 21:04:25.930368 28567 catalog_manager.cc:1347] Generated new cluster ID: f4fafe6c1eec45fe8bbb601b23703c2b
I20260812 21:04:25.930421 28567 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 21:04:25.937709 28567 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 21:04:25.938341 28567 catalog_manager.cc:1504] Loading token signing keys...
I20260812 21:04:25.942746 28567 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b1b45b68856a4014be8cbe76cfffe14a: Generated new TSK 0
I20260812 21:04:25.942932 28567 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 21:04:25.944267 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786568607354921-12282-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786568607354921-12282-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786568607354921-12282-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786568607354921-12282-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.254.129:0
--local_ip_for_outbound_sockets=127.11.254.129
--webserver_interface=127.11.254.129
--webserver_port=0
--tserver_master_addrs=127.11.254.190:40569
--builtin_ntp_servers=127.11.254.148:42629
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260812 21:04:26.029059 28584 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:04:26.029271 28584 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:04:26.029294 28584 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:04:26.030529 28584 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:04:26.030573 28584 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:04:26.030598 28584 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.254.129
I20260812 21:04:26.031953 28584 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.254.148:42629
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786568607354921-12282-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786568607354921-12282-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.254.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786568607354921-12282-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.254.129
--webserver_port=0
--tserver_master_addrs=127.11.254.190:40569
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28584
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.254.129
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786568607354921-12282-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:04:26.032182 28584 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:04:26.032516 28584 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:04:26.037673 28590 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:04:26.037770 28584 server_base.cc:1034] running on GCE node
W20260812 21:04:26.037673 28591 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:04:26.038053 28594 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:04:26.038322 28584 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:04:26.038635 28584 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:04:26.039793 28584 hybrid_clock.cc:648] HybridClock initialized: now 1786568666039775 us; error 34 us; skew 500 ppm
I20260812 21:04:26.041239 28584 webserver.cc:460] Webserver started at http://127.11.254.129:39429/ using document root <none> and password file <none>
I20260812 21:04:26.041533 28584 fs_manager.cc:359] Metadata directory not provided
I20260812 21:04:26.041594 28584 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:04:26.041738 28584 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:04:26.043296 28584 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786568607354921-12282-0/minicluster-data/ts-0/data/instance:
uuid: "de7acfe88cda451cbd8895f4d9fa107b"
format_stamp: "Formatted at 2026-08-12 21:04:26 on dist-test-slave-vpvm"
server_key: "884b6ed2e9e78dc0e103b3eb25e24156"
server_key_iv: "7b2c708b2634aa767d12a2c3070cd390"
server_key_version: "encryptionkey@0"
I20260812 21:04:26.043745 28584 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786568607354921-12282-0/minicluster-data/ts-0/wal/instance:
uuid: "de7acfe88cda451cbd8895f4d9fa107b"
format_stamp: "Formatted at 2026-08-12 21:04:26 on dist-test-slave-vpvm"
server_key: "884b6ed2e9e78dc0e103b3eb25e24156"
server_key_iv: "7b2c708b2634aa767d12a2c3070cd390"
server_key_version: "encryptionkey@0"
I20260812 21:04:26.045389 28584 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 21:04:26.046360 28603 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:04:26.046528 28584 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 21:04:26.046618 28584 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786568607354921-12282-0/minicluster-data/ts-0/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786568607354921-12282-0/minicluster-data/ts-0/wal
uuid: "de7acfe88cda451cbd8895f4d9fa107b"
format_stamp: "Formatted at 2026-08-12 21:04:26 on dist-test-slave-vpvm"
server_key: "884b6ed2e9e78dc0e103b3eb25e24156"
server_key_iv: "7b2c708b2634aa767d12a2c3070cd390"
server_key_version: "encryptionkey@0"
I20260812 21:04:26.046693 28584 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786568607354921-12282-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786568607354921-12282-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786568607354921-12282-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:04:26.060158 28584 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:04:26.060475 28584 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:04:26.060609 28584 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:04:26.060839 28584 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:04:26.061180 28584 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:04:26.061214 28584 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:04:26.061237 28584 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:04:26.061261 28584 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:04:26.067086 28584 rpc_server.cc:307] RPC server started. Bound to: 127.11.254.129:40063
I20260812 21:04:26.067175 28783 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.254.129:40063 every 8 connection(s)
I20260812 21:04:26.067467 28584 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786568607354921-12282-0/minicluster-data/ts-0/data/info.pb
I20260812 21:04:26.070143 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 28584
I20260812 21:04:26.070242 12282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786568607354921-12282-0/minicluster-data/ts-0/wal/instance
I20260812 21:04:26.070447 12282 external_mini_cluster.cc:1428] Setting key a26144f8c3cda7eacb2999c10fc86b7c
I20260812 21:04:26.071383 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786568607354921-12282-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786568607354921-12282-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786568607354921-12282-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786568607354921-12282-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.254.130:0
--local_ip_for_outbound_sockets=127.11.254.130
--webserver_interface=127.11.254.130
--webserver_port=0
--tserver_master_addrs=127.11.254.190:40569
--builtin_ntp_servers=127.11.254.148:42629
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260812 21:04:26.073276 28546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:26.068178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.129:46481 (local address 127.11.254.190:40569)
0812 21:04:26.068239 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0812 21:04:26.068242 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 21:04:26.068523 (+ 281us) server_negotiation.cc:407] Connection header received
0812 21:04:26.069089 (+ 566us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:04:26.069091 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:04:26.069121 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:04:26.069176 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:04:26.069881 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:26.070644 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:26.071849 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:26.072056 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:26.072119 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:26.072989 (+ 870us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:04:26.072990 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:04:26.072991 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 21:04:26.072996 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:04:26.073016 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 21:04:26.073020 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:04:26.073046 (+ 26us) server_negotiation.cc:1035] Waiting for connection context
0812 21:04:26.073178 (+ 132us) server_negotiation.cc:299] Negotiation successful
0812 21:04:26.073211 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260812 21:04:26.073515 28791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:26.068174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:40569 (local address 127.11.254.129:46481)
0812 21:04:26.068403 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0812 21:04:26.068444 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 21:04:26.068983 (+ 539us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:04:26.069251 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:04:26.069255 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:04:26.069266 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:04:26.069790 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:26.069795 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:26.070744 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:26.070745 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:26.071762 (+ 1017us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:26.071765 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:26.072126 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:26.072127 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:26.072267 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:26.072802 (+ 535us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 21:04:26.072803 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:04:26.072845 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 21:04:26.072879 (+ 34us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:04:26.073083 (+ 204us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:04:26.073088 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:04:26.073091 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 21:04:26.073159 (+ 68us) client_negotiation.cc:240] Negotiation successful
0812 21:04:26.073282 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":82,"threads_started":1}
I20260812 21:04:26.074038 28784 heartbeater.cc:347] Connected to a master server at 127.11.254.190:40569
I20260812 21:04:26.074144 28784 heartbeater.cc:464] Registering TS with master...
I20260812 21:04:26.074340 28784 heartbeater.cc:511] Master 127.11.254.190:40569 requested a full tablet report, sending...
I20260812 21:04:26.074929 28481 ts_manager.cc:194] Registered new tserver with Master: de7acfe88cda451cbd8895f4d9fa107b (127.11.254.129:40063)
I20260812 21:04:26.075883 28481 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.254.129:46481
W20260812 21:04:26.154460 28792 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:04:26.154687 28792 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:04:26.154708 28792 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:04:26.155820 28792 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:04:26.155854 28792 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:04:26.155879 28792 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.254.130
I20260812 21:04:26.157167 28792 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.254.148:42629
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786568607354921-12282-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786568607354921-12282-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.254.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786568607354921-12282-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.254.130
--webserver_port=0
--tserver_master_addrs=127.11.254.190:40569
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28792
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.254.130
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786568607354921-12282-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:04:26.157392 28792 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:04:26.157677 28792 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:04:26.162971 28802 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:04:26.162946 28801 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:04:26.162952 28805 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:04:26.163144 28792 server_base.cc:1034] running on GCE node
I20260812 21:04:26.163407 28792 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:04:26.163703 28792 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:04:26.164884 28792 hybrid_clock.cc:648] HybridClock initialized: now 1786568666164863 us; error 37 us; skew 500 ppm
I20260812 21:04:26.166152 28792 webserver.cc:460] Webserver started at http://127.11.254.130:39489/ using document root <none> and password file <none>
I20260812 21:04:26.166368 28792 fs_manager.cc:359] Metadata directory not provided
I20260812 21:04:26.166410 28792 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:04:26.166545 28792 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:04:26.167625 28792 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786568607354921-12282-0/minicluster-data/ts-1/data/instance:
uuid: "67215130f0454d81b4f1ede2a38d2d32"
format_stamp: "Formatted at 2026-08-12 21:04:26 on dist-test-slave-vpvm"
server_key: "5c2dfa3ee861050b2d8fcf22285a8e42"
server_key_iv: "c45474d4eee4e61b0eea05bea1a821d0"
server_key_version: "encryptionkey@0"
I20260812 21:04:26.167932 28792 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786568607354921-12282-0/minicluster-data/ts-1/wal/instance:
uuid: "67215130f0454d81b4f1ede2a38d2d32"
format_stamp: "Formatted at 2026-08-12 21:04:26 on dist-test-slave-vpvm"
server_key: "5c2dfa3ee861050b2d8fcf22285a8e42"
server_key_iv: "c45474d4eee4e61b0eea05bea1a821d0"
server_key_version: "encryptionkey@0"
I20260812 21:04:26.169164 28792 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 21:04:26.169842 28814 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:04:26.169992 28792 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 21:04:26.170074 28792 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786568607354921-12282-0/minicluster-data/ts-1/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786568607354921-12282-0/minicluster-data/ts-1/wal
uuid: "67215130f0454d81b4f1ede2a38d2d32"
format_stamp: "Formatted at 2026-08-12 21:04:26 on dist-test-slave-vpvm"
server_key: "5c2dfa3ee861050b2d8fcf22285a8e42"
server_key_iv: "c45474d4eee4e61b0eea05bea1a821d0"
server_key_version: "encryptionkey@0"
I20260812 21:04:26.170146 28792 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786568607354921-12282-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786568607354921-12282-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786568607354921-12282-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:04:26.194204 28792 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:04:26.194551 28792 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:04:26.194679 28792 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:04:26.194916 28792 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:04:26.195268 28792 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:04:26.195305 28792 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:04:26.195329 28792 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:04:26.195353 28792 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:04:26.201014 28792 rpc_server.cc:307] RPC server started. Bound to: 127.11.254.130:46575
I20260812 21:04:26.201081 28984 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.254.130:46575 every 8 connection(s)
I20260812 21:04:26.201406 28792 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786568607354921-12282-0/minicluster-data/ts-1/data/info.pb
I20260812 21:04:26.206671 28546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:26.202142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.130:40127 (local address 127.11.254.190:40569)
0812 21:04:26.202206 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0812 21:04:26.202208 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:04:26.202454 (+ 246us) server_negotiation.cc:407] Connection header received
0812 21:04:26.202991 (+ 537us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:04:26.202992 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:04:26.203004 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:04:26.203062 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:04:26.203758 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:26.204294 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:26.205278 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:26.205529 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:26.205587 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:26.206423 (+ 836us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:04:26.206424 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:04:26.206425 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 21:04:26.206429 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:04:26.206446 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 21:04:26.206450 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:04:26.206495 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0812 21:04:26.206568 (+ 73us) server_negotiation.cc:299] Negotiation successful
0812 21:04:26.206592 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260812 21:04:26.206861 28987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:26.202147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:40569 (local address 127.11.254.130:40127)
0812 21:04:26.202348 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0812 21:04:26.202383 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0812 21:04:26.202877 (+ 494us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:04:26.203136 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:04:26.203139 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:04:26.203148 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:04:26.203656 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:26.203660 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:26.204376 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:26.204377 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:26.205200 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:26.205203 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:26.205598 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:26.205599 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:26.205717 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:26.206278 (+ 561us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 21:04:26.206278 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:04:26.206308 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 21:04:26.206332 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:04:26.206507 (+ 175us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:04:26.206510 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:04:26.206513 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 21:04:26.206565 (+ 52us) client_negotiation.cc:240] Negotiation successful
0812 21:04:26.206652 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":88,"threads_started":1}
I20260812 21:04:26.207360 28985 heartbeater.cc:347] Connected to a master server at 127.11.254.190:40569
I20260812 21:04:26.207456 28985 heartbeater.cc:464] Registering TS with master...
I20260812 21:04:26.207639 28985 heartbeater.cc:511] Master 127.11.254.190:40569 requested a full tablet report, sending...
I20260812 21:04:26.207976 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 28792
I20260812 21:04:26.208022 28481 ts_manager.cc:194] Registered new tserver with Master: 67215130f0454d81b4f1ede2a38d2d32 (127.11.254.130:46575)
I20260812 21:04:26.208060 12282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786568607354921-12282-0/minicluster-data/ts-1/wal/instance
I20260812 21:04:26.208266 12282 external_mini_cluster.cc:1428] Setting key 7607d014c24b2f2107a5e5080270a468
I20260812 21:04:26.208400 28481 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.254.130:40127
I20260812 21:04:26.209219 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786568607354921-12282-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786568607354921-12282-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786568607354921-12282-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786568607354921-12282-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.254.131:0
--local_ip_for_outbound_sockets=127.11.254.131
--webserver_interface=127.11.254.131
--webserver_port=0
--tserver_master_addrs=127.11.254.190:40569
--builtin_ntp_servers=127.11.254.148:42629
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260812 21:04:26.295817 28988 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:04:26.296031 28988 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:04:26.296051 28988 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:04:26.297178 28988 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:04:26.297214 28988 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:04:26.297245 28988 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.254.131
I20260812 21:04:26.298605 28988 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.254.148:42629
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786568607354921-12282-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786568607354921-12282-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.254.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786568607354921-12282-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.254.131
--webserver_port=0
--tserver_master_addrs=127.11.254.190:40569
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28988
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.254.131
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786568607354921-12282-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:04:26.298833 28988 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:04:26.299108 28988 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:04:26.304190 28995 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:04:26.304229 28998 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:04:26.304246 28996 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:04:26.304352 28988 server_base.cc:1034] running on GCE node
I20260812 21:04:26.304711 28988 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:04:26.304932 28988 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:04:26.306025 28988 hybrid_clock.cc:648] HybridClock initialized: now 1786568666306008 us; error 32 us; skew 500 ppm
I20260812 21:04:26.307262 28988 webserver.cc:460] Webserver started at http://127.11.254.131:38013/ using document root <none> and password file <none>
I20260812 21:04:26.307497 28988 fs_manager.cc:359] Metadata directory not provided
I20260812 21:04:26.307557 28988 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:04:26.307698 28988 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:04:26.308789 28988 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786568607354921-12282-0/minicluster-data/ts-2/data/instance:
uuid: "66ec61111a6846eba7cb6718d1114b4a"
format_stamp: "Formatted at 2026-08-12 21:04:26 on dist-test-slave-vpvm"
server_key: "3341082ced01da46a311427f1cc7e932"
server_key_iv: "d70e27233ea7b57e7db6bf0e29e0d728"
server_key_version: "encryptionkey@0"
I20260812 21:04:26.309140 28988 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786568607354921-12282-0/minicluster-data/ts-2/wal/instance:
uuid: "66ec61111a6846eba7cb6718d1114b4a"
format_stamp: "Formatted at 2026-08-12 21:04:26 on dist-test-slave-vpvm"
server_key: "3341082ced01da46a311427f1cc7e932"
server_key_iv: "d70e27233ea7b57e7db6bf0e29e0d728"
server_key_version: "encryptionkey@0"
I20260812 21:04:26.310472 28988 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 21:04:26.311182 29005 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:04:26.311349 28988 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 21:04:26.311437 28988 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786568607354921-12282-0/minicluster-data/ts-2/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786568607354921-12282-0/minicluster-data/ts-2/wal
uuid: "66ec61111a6846eba7cb6718d1114b4a"
format_stamp: "Formatted at 2026-08-12 21:04:26 on dist-test-slave-vpvm"
server_key: "3341082ced01da46a311427f1cc7e932"
server_key_iv: "d70e27233ea7b57e7db6bf0e29e0d728"
server_key_version: "encryptionkey@0"
I20260812 21:04:26.311498 28988 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786568607354921-12282-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786568607354921-12282-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786568607354921-12282-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:04:26.324131 28988 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:04:26.324416 28988 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:04:26.324522 28988 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:04:26.324714 28988 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:04:26.325052 28988 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:04:26.325088 28988 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:04:26.325131 28988 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:04:26.325155 28988 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:04:26.330755 28988 rpc_server.cc:307] RPC server started. Bound to: 127.11.254.131:39903
I20260812 21:04:26.330821 29185 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.254.131:39903 every 8 connection(s)
I20260812 21:04:26.331147 28988 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786568607354921-12282-0/minicluster-data/ts-2/data/info.pb
I20260812 21:04:26.334977 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 28988
I20260812 21:04:26.335085 12282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786568607354921-12282-0/minicluster-data/ts-2/wal/instance
I20260812 21:04:26.335285 12282 external_mini_cluster.cc:1428] Setting key 196b2206c72bf06c893b685536edc318
I20260812 21:04:26.336684 28546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:26.331828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.131:34107 (local address 127.11.254.190:40569)
0812 21:04:26.331893 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0812 21:04:26.331895 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:04:26.332128 (+ 233us) server_negotiation.cc:407] Connection header received
0812 21:04:26.332705 (+ 577us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:04:26.332707 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:04:26.332718 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:04:26.332791 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:04:26.333530 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:26.334101 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:26.335218 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:26.335435 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:26.335500 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:26.336423 (+ 923us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:04:26.336424 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:04:26.336424 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 21:04:26.336429 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:04:26.336446 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 21:04:26.336450 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:04:26.336493 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0812 21:04:26.336567 (+ 74us) server_negotiation.cc:299] Negotiation successful
0812 21:04:26.336596 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260812 21:04:26.336885 29194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:26.331820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:40569 (local address 127.11.254.131:34107)
0812 21:04:26.332011 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0812 21:04:26.332046 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0812 21:04:26.332539 (+ 493us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:04:26.332867 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:04:26.332870 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:04:26.332880 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:04:26.333438 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:26.333443 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:26.334182 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:26.334182 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:26.335105 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:26.335119 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:26.335528 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:26.335529 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:26.335689 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:26.336270 (+ 581us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 21:04:26.336271 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:04:26.336311 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 21:04:26.336340 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:04:26.336501 (+ 161us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:04:26.336506 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:04:26.336509 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 21:04:26.336570 (+ 61us) client_negotiation.cc:240] Negotiation successful
0812 21:04:26.336678 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":94,"threads_started":1}
I20260812 21:04:26.337289 29191 heartbeater.cc:347] Connected to a master server at 127.11.254.190:40569
I20260812 21:04:26.337378 29191 heartbeater.cc:464] Registering TS with master...
I20260812 21:04:26.337555 29191 heartbeater.cc:511] Master 127.11.254.190:40569 requested a full tablet report, sending...
I20260812 21:04:26.337893 28481 ts_manager.cc:194] Registered new tserver with Master: 66ec61111a6846eba7cb6718d1114b4a (127.11.254.131:39903)
I20260812 21:04:26.338287 28481 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.254.131:34107
I20260812 21:04:26.347144 12282 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260812 21:04:26.349005 28546 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:04:26.348514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56926 (local address 127.11.254.190:40569)
0812 21:04:26.348576 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0812 21:04:26.348579 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 21:04:26.348680 (+ 101us) server_negotiation.cc:407] Connection header received
0812 21:04:26.348735 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:04:26.348736 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:04:26.348745 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:04:26.348791 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:04:26.348938 (+ 147us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56926: BlockingRecv error: recv got EOF from 127.0.0.1:56926 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
I20260812 21:04:26.349072 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 28584
I20260812 21:04:26.363706 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 28792
I20260812 21:04:26.371417 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 28988
I20260812 21:04:26.383280 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 28436
2026-08-12T21:04:26Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (612 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-12T21:04:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T21:04:26Z Disabled control of system clock
I20260812 21:04:26.415786 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786568607354921-12282-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786568607354921-12282-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786568607354921-12282-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.254.190:41747
--webserver_interface=127.11.254.190
--webserver_port=0
--builtin_ntp_servers=127.11.254.148:37843
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.254.190:41747
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260812 21:04:26.501922 29207 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:04:26.502195 29207 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:04:26.502218 29207 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:04:26.503408 29207 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 21:04:26.503450 29207 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:04:26.503461 29207 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:04:26.503484 29207 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 21:04:26.503507 29207 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 21:04:26.504884 29207 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.254.148:37843
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786568607354921-12282-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786568607354921-12282-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.254.190:41747
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.254.190:41747
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.254.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29207
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786568607354921-12282-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:04:26.505133 29207 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:04:26.505430 29207 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 21:04:26.510480 29207 server_base.cc:1034] running on GCE node
W20260812 21:04:26.510411 29216 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:04:26.510476 29215 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:04:26.510411 29218 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:04:26.511018 29207 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:04:26.511317 29207 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:04:26.512465 29207 hybrid_clock.cc:648] HybridClock initialized: now 1786568666512451 us; error 30 us; skew 500 ppm
I20260812 21:04:26.513732 29207 webserver.cc:460] Webserver started at http://127.11.254.190:37121/ using document root <none> and password file <none>
I20260812 21:04:26.514009 29207 fs_manager.cc:359] Metadata directory not provided
I20260812 21:04:26.514066 29207 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:04:26.514205 29207 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:04:26.515349 29207 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786568607354921-12282-0/minicluster-data/master-0/data/instance:
uuid: "bc9f8948ee2446689abb02df40023da0"
format_stamp: "Formatted at 2026-08-12 21:04:26 on dist-test-slave-vpvm"
server_key: "f9bffcb47196110c84f780c34ef93ea6"
server_key_iv: "4b4cd6380a96044223a8c70c53167cc2"
server_key_version: "encryptionkey@0"
I20260812 21:04:26.515717 29207 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786568607354921-12282-0/minicluster-data/master-0/wal/instance:
uuid: "bc9f8948ee2446689abb02df40023da0"
format_stamp: "Formatted at 2026-08-12 21:04:26 on dist-test-slave-vpvm"
server_key: "f9bffcb47196110c84f780c34ef93ea6"
server_key_iv: "4b4cd6380a96044223a8c70c53167cc2"
server_key_version: "encryptionkey@0"
I20260812 21:04:26.517016 29207 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 21:04:26.517742 29229 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:04:26.517889 29207 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 21:04:26.518004 29207 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786568607354921-12282-0/minicluster-data/master-0/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786568607354921-12282-0/minicluster-data/master-0/wal
uuid: "bc9f8948ee2446689abb02df40023da0"
format_stamp: "Formatted at 2026-08-12 21:04:26 on dist-test-slave-vpvm"
server_key: "f9bffcb47196110c84f780c34ef93ea6"
server_key_iv: "4b4cd6380a96044223a8c70c53167cc2"
server_key_version: "encryptionkey@0"
I20260812 21:04:26.518074 29207 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786568607354921-12282-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786568607354921-12282-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786568607354921-12282-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:04:26.521504 29207 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:04:26.521799 29207 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:04:26.521932 29207 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:04:26.525846 29207 rpc_server.cc:307] RPC server started. Bound to: 127.11.254.190:41747
I20260812 21:04:26.525892 29307 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.254.190:41747 every 8 connection(s)
I20260812 21:04:26.526237 29207 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
I20260812 21:04:26.526945 29310 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:04:26.530014 29310 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bc9f8948ee2446689abb02df40023da0: Bootstrap starting.
I20260812 21:04:26.530406 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 29207
I20260812 21:04:26.530479 12282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786568607354921-12282-0/minicluster-data/master-0/wal/instance
I20260812 21:04:26.530653 12282 external_mini_cluster.cc:1428] Setting key d395d69e5bbc3b26aeddaae964d3148c
I20260812 21:04:26.530701 29310 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bc9f8948ee2446689abb02df40023da0: Neither blocks nor log segments found. Creating new log.
I20260812 21:04:26.530994 29310 log.cc:824] T 00000000000000000000000000000000 P bc9f8948ee2446689abb02df40023da0: Log is configured to *not* fsync() on all Append() calls
I20260812 21:04:26.531716 29310 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bc9f8948ee2446689abb02df40023da0: No bootstrap required, opened a new log
I20260812 21:04:26.533226 29315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:26.531189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44130 (local address 127.11.254.190:41747)
0812 21:04:26.531397 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0812 21:04:26.531405 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 21:04:26.531429 (+ 24us) server_negotiation.cc:407] Connection header received
0812 21:04:26.531907 (+ 478us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:04:26.531915 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:04:26.531957 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:04:26.532093 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:04:26.532822 (+ 729us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:04:26.532825 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:04:26.532825 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 21:04:26.532829 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:04:26.532870 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 21:04:26.532879 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:04:26.532953 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0812 21:04:26.533006 (+ 53us) server_negotiation.cc:299] Negotiation successful
0812 21:04:26.533065 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":81,"threads_started":1}
I20260812 21:04:26.533422 29310 raft_consensus.cc:348] T 00000000000000000000000000000000 P bc9f8948ee2446689abb02df40023da0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc9f8948ee2446689abb02df40023da0" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 41747 } }
I20260812 21:04:26.533597 29310 raft_consensus.cc:374] T 00000000000000000000000000000000 P bc9f8948ee2446689abb02df40023da0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:04:26.533635 29310 raft_consensus.cc:729] T 00000000000000000000000000000000 P bc9f8948ee2446689abb02df40023da0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc9f8948ee2446689abb02df40023da0, State: Initialized, Role: FOLLOWER
I20260812 21:04:26.533758 29310 consensus_queue.cc:260] T 00000000000000000000000000000000 P bc9f8948ee2446689abb02df40023da0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc9f8948ee2446689abb02df40023da0" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 41747 } }
I20260812 21:04:26.533839 29310 raft_consensus.cc:388] T 00000000000000000000000000000000 P bc9f8948ee2446689abb02df40023da0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 21:04:26.533872 29310 raft_consensus.cc:482] T 00000000000000000000000000000000 P bc9f8948ee2446689abb02df40023da0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 21:04:26.533917 29310 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bc9f8948ee2446689abb02df40023da0 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:04:26.534797 29310 raft_consensus.cc:504] T 00000000000000000000000000000000 P bc9f8948ee2446689abb02df40023da0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc9f8948ee2446689abb02df40023da0" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 41747 } }
I20260812 21:04:26.534898 29310 leader_election.cc:302] T 00000000000000000000000000000000 P bc9f8948ee2446689abb02df40023da0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bc9f8948ee2446689abb02df40023da0; no voters:
I20260812 21:04:26.535056 29310 leader_election.cc:288] T 00000000000000000000000000000000 P bc9f8948ee2446689abb02df40023da0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 21:04:26.535101 29316 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bc9f8948ee2446689abb02df40023da0 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:04:26.535249 29316 raft_consensus.cc:686] T 00000000000000000000000000000000 P bc9f8948ee2446689abb02df40023da0 [term 1 LEADER]: Becoming Leader. State: Replica: bc9f8948ee2446689abb02df40023da0, State: Running, Role: LEADER
I20260812 21:04:26.535322 29310 sys_catalog.cc:564] T 00000000000000000000000000000000 P bc9f8948ee2446689abb02df40023da0 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 21:04:26.535352 29316 consensus_queue.cc:237] T 00000000000000000000000000000000 P bc9f8948ee2446689abb02df40023da0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc9f8948ee2446689abb02df40023da0" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 41747 } }
I20260812 21:04:26.535717 29319 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc9f8948ee2446689abb02df40023da0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bc9f8948ee2446689abb02df40023da0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc9f8948ee2446689abb02df40023da0" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 41747 } } }
I20260812 21:04:26.535740 29320 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc9f8948ee2446689abb02df40023da0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bc9f8948ee2446689abb02df40023da0. Latest consensus state: current_term: 1 leader_uuid: "bc9f8948ee2446689abb02df40023da0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc9f8948ee2446689abb02df40023da0" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 41747 } } }
I20260812 21:04:26.535867 29319 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc9f8948ee2446689abb02df40023da0 [sys.catalog]: This master's current role is: LEADER
I20260812 21:04:26.535887 29320 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc9f8948ee2446689abb02df40023da0 [sys.catalog]: This master's current role is: LEADER
I20260812 21:04:26.536187 29337 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 21:04:26.536721 29337 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 21:04:26.538278 29337 catalog_manager.cc:1347] Generated new cluster ID: 36044c3b53e14a6e8a846020c255c283
I20260812 21:04:26.538335 29337 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 21:04:26.543805 29337 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 21:04:26.543987 29337 catalog_manager.cc:1504] Loading token signing keys...
I20260812 21:04:26.559701 29337 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bc9f8948ee2446689abb02df40023da0: Generated new TSK 0
I20260812 21:04:26.559911 29337 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 21:04:26.564096 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786568607354921-12282-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786568607354921-12282-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786568607354921-12282-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786568607354921-12282-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.254.129:0
--local_ip_for_outbound_sockets=127.11.254.129
--webserver_interface=127.11.254.129
--webserver_port=0
--tserver_master_addrs=127.11.254.190:41747
--builtin_ntp_servers=127.11.254.148:37843
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260812 21:04:26.647892 29352 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:04:26.648126 29352 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:04:26.648162 29352 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:04:26.649379 29352 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:04:26.649421 29352 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:04:26.649458 29352 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.254.129
I20260812 21:04:26.650909 29352 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.254.148:37843
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786568607354921-12282-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786568607354921-12282-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.254.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786568607354921-12282-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.254.129
--webserver_port=0
--tserver_master_addrs=127.11.254.190:41747
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29352
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.254.129
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786568607354921-12282-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:04:26.651211 29352 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:04:26.651527 29352 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:04:26.657210 29361 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:04:26.657214 29359 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:04:26.657292 29352 server_base.cc:1034] running on GCE node
W20260812 21:04:26.657509 29363 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:04:26.657788 29352 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:04:26.658106 29352 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:04:26.659294 29352 hybrid_clock.cc:648] HybridClock initialized: now 1786568666659275 us; error 30 us; skew 500 ppm
I20260812 21:04:26.660833 29352 webserver.cc:460] Webserver started at http://127.11.254.129:37829/ using document root <none> and password file <none>
I20260812 21:04:26.661128 29352 fs_manager.cc:359] Metadata directory not provided
I20260812 21:04:26.661192 29352 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:04:26.661326 29352 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:04:26.662845 29352 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786568607354921-12282-0/minicluster-data/ts-0/data/instance:
uuid: "c1a18fe3db1747409d6fb73af2326b65"
format_stamp: "Formatted at 2026-08-12 21:04:26 on dist-test-slave-vpvm"
server_key: "389e818606985694bc70057b1dd98202"
server_key_iv: "96356b9a4c242dc97c230fc35275ca06"
server_key_version: "encryptionkey@0"
I20260812 21:04:26.663251 29352 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786568607354921-12282-0/minicluster-data/ts-0/wal/instance:
uuid: "c1a18fe3db1747409d6fb73af2326b65"
format_stamp: "Formatted at 2026-08-12 21:04:26 on dist-test-slave-vpvm"
server_key: "389e818606985694bc70057b1dd98202"
server_key_iv: "96356b9a4c242dc97c230fc35275ca06"
server_key_version: "encryptionkey@0"
I20260812 21:04:26.664602 29352 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260812 21:04:26.665392 29372 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:04:26.665562 29352 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 21:04:26.665670 29352 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786568607354921-12282-0/minicluster-data/ts-0/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786568607354921-12282-0/minicluster-data/ts-0/wal
uuid: "c1a18fe3db1747409d6fb73af2326b65"
format_stamp: "Formatted at 2026-08-12 21:04:26 on dist-test-slave-vpvm"
server_key: "389e818606985694bc70057b1dd98202"
server_key_iv: "96356b9a4c242dc97c230fc35275ca06"
server_key_version: "encryptionkey@0"
I20260812 21:04:26.665747 29352 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786568607354921-12282-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786568607354921-12282-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786568607354921-12282-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:04:26.669284 29352 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:04:26.669613 29352 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:04:26.669755 29352 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:04:26.669993 29352 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:04:26.670316 29352 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:04:26.670351 29352 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:04:26.670378 29352 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:04:26.670396 29352 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:04:26.676098 29352 rpc_server.cc:307] RPC server started. Bound to: 127.11.254.129:35909
I20260812 21:04:26.676154 29557 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.254.129:35909 every 8 connection(s)
I20260812 21:04:26.676472 29352 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786568607354921-12282-0/minicluster-data/ts-0/data/info.pb
I20260812 21:04:26.679340 29315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:26.677222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.129:35899 (local address 127.11.254.190:41747)
0812 21:04:26.677285 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0812 21:04:26.677287 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:04:26.677505 (+ 218us) server_negotiation.cc:407] Connection header received
0812 21:04:26.678003 (+ 498us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:04:26.678004 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:04:26.678009 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:04:26.678068 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:04:26.679154 (+ 1086us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:04:26.679155 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:04:26.679155 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 21:04:26.679159 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:04:26.679174 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 21:04:26.679178 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:04:26.679224 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0812 21:04:26.679274 (+ 50us) server_negotiation.cc:299] Negotiation successful
0812 21:04:26.679285 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260812 21:04:26.679525 29560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:26.677195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:41747 (local address 127.11.254.129:35899)
0812 21:04:26.677382 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0812 21:04:26.677424 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 21:04:26.677895 (+ 471us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:04:26.678134 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:04:26.678137 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:04:26.678146 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:04:26.679009 (+ 863us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 21:04:26.679010 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:04:26.679038 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 21:04:26.679068 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:04:26.679222 (+ 154us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:04:26.679224 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:04:26.679226 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 21:04:26.679276 (+ 50us) client_negotiation.cc:240] Negotiation successful
0812 21:04:26.679324 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":86,"threads_started":1}
I20260812 21:04:26.679759 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 29352
I20260812 21:04:26.679857 12282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786568607354921-12282-0/minicluster-data/ts-0/wal/instance
I20260812 21:04:26.680042 12282 external_mini_cluster.cc:1428] Setting key 12b4abac2cb27cbe965a2f5137f3a828
I20260812 21:04:26.680172 29558 heartbeater.cc:347] Connected to a master server at 127.11.254.190:41747
I20260812 21:04:26.680275 29558 heartbeater.cc:464] Registering TS with master...
I20260812 21:04:26.680459 29558 heartbeater.cc:511] Master 127.11.254.190:41747 requested a full tablet report, sending...
I20260812 21:04:26.681032 29257 ts_manager.cc:194] Registered new tserver with Master: c1a18fe3db1747409d6fb73af2326b65 (127.11.254.129:35909)
I20260812 21:04:26.681097 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786568607354921-12282-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786568607354921-12282-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786568607354921-12282-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786568607354921-12282-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.254.130:0
--local_ip_for_outbound_sockets=127.11.254.130
--webserver_interface=127.11.254.130
--webserver_port=0
--tserver_master_addrs=127.11.254.190:41747
--builtin_ntp_servers=127.11.254.148:37843
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260812 21:04:26.764062 29561 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:04:26.764307 29561 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:04:26.764331 29561 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:04:26.765576 29561 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:04:26.765631 29561 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:04:26.765659 29561 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.254.130
I20260812 21:04:26.767128 29561 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.254.148:37843
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786568607354921-12282-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786568607354921-12282-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.254.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786568607354921-12282-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.254.130
--webserver_port=0
--tserver_master_addrs=127.11.254.190:41747
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29561
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.254.130
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786568607354921-12282-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:04:26.767390 29561 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:04:26.767700 29561 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:04:26.773190 29572 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:04:26.773190 29576 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:04:26.773387 29574 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:04:26.773671 29561 server_base.cc:1034] running on GCE node
I20260812 21:04:26.773921 29561 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:04:26.774201 29561 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:04:26.775349 29561 hybrid_clock.cc:648] HybridClock initialized: now 1786568666775332 us; error 29 us; skew 500 ppm
I20260812 21:04:26.776541 29561 webserver.cc:460] Webserver started at http://127.11.254.130:40017/ using document root <none> and password file <none>
I20260812 21:04:26.776764 29561 fs_manager.cc:359] Metadata directory not provided
I20260812 21:04:26.776814 29561 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:04:26.776939 29561 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:04:26.778116 29561 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786568607354921-12282-0/minicluster-data/ts-1/data/instance:
uuid: "afd42b27cfba4ab2a0da6d6936140716"
format_stamp: "Formatted at 2026-08-12 21:04:26 on dist-test-slave-vpvm"
server_key: "61e2303461d884103428001bd58b0d36"
server_key_iv: "e0e1478d5feedb708c9e62576916cdb3"
server_key_version: "encryptionkey@0"
I20260812 21:04:26.778491 29561 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786568607354921-12282-0/minicluster-data/ts-1/wal/instance:
uuid: "afd42b27cfba4ab2a0da6d6936140716"
format_stamp: "Formatted at 2026-08-12 21:04:26 on dist-test-slave-vpvm"
server_key: "61e2303461d884103428001bd58b0d36"
server_key_iv: "e0e1478d5feedb708c9e62576916cdb3"
server_key_version: "encryptionkey@0"
I20260812 21:04:26.779770 29561 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 21:04:26.780445 29587 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:04:26.780704 29561 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 21:04:26.780807 29561 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786568607354921-12282-0/minicluster-data/ts-1/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786568607354921-12282-0/minicluster-data/ts-1/wal
uuid: "afd42b27cfba4ab2a0da6d6936140716"
format_stamp: "Formatted at 2026-08-12 21:04:26 on dist-test-slave-vpvm"
server_key: "61e2303461d884103428001bd58b0d36"
server_key_iv: "e0e1478d5feedb708c9e62576916cdb3"
server_key_version: "encryptionkey@0"
I20260812 21:04:26.780879 29561 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786568607354921-12282-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786568607354921-12282-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786568607354921-12282-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:04:26.784531 29561 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:04:26.784791 29561 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:04:26.784921 29561 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:04:26.785117 29561 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:04:26.785432 29561 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:04:26.785466 29561 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:04:26.785493 29561 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:04:26.785517 29561 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:04:26.791460 29561 rpc_server.cc:307] RPC server started. Bound to: 127.11.254.130:39819
I20260812 21:04:26.791499 29766 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.254.130:39819 every 8 connection(s)
I20260812 21:04:26.791828 29561 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786568607354921-12282-0/minicluster-data/ts-1/data/info.pb
I20260812 21:04:26.794828 29315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:26.792602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.130:36955 (local address 127.11.254.190:41747)
0812 21:04:26.792716 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0812 21:04:26.792718 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:04:26.792934 (+ 216us) server_negotiation.cc:407] Connection header received
0812 21:04:26.793403 (+ 469us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:04:26.793404 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:04:26.793411 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:04:26.793475 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:04:26.794627 (+ 1152us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:04:26.794628 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:04:26.794628 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 21:04:26.794632 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:04:26.794646 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 21:04:26.794649 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:04:26.794694 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0812 21:04:26.794758 (+ 64us) server_negotiation.cc:299] Negotiation successful
0812 21:04:26.794770 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260812 21:04:26.795009 29774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:26.792538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:41747 (local address 127.11.254.130:36955)
0812 21:04:26.792792 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0812 21:04:26.792834 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 21:04:26.793316 (+ 482us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:04:26.793542 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:04:26.793545 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:04:26.793554 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:04:26.794485 (+ 931us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 21:04:26.794486 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:04:26.794516 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 21:04:26.794546 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:04:26.794703 (+ 157us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:04:26.794705 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:04:26.794708 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 21:04:26.794764 (+ 56us) client_negotiation.cc:240] Negotiation successful
0812 21:04:26.794815 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":134,"threads_started":1}
I20260812 21:04:26.795436 29768 heartbeater.cc:347] Connected to a master server at 127.11.254.190:41747
I20260812 21:04:26.795526 29768 heartbeater.cc:464] Registering TS with master...
I20260812 21:04:26.795652 29768 heartbeater.cc:511] Master 127.11.254.190:41747 requested a full tablet report, sending...
I20260812 21:04:26.795995 29257 ts_manager.cc:194] Registered new tserver with Master: afd42b27cfba4ab2a0da6d6936140716 (127.11.254.130:39819)
I20260812 21:04:26.796502 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 29561
I20260812 21:04:26.796602 12282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786568607354921-12282-0/minicluster-data/ts-1/wal/instance
I20260812 21:04:26.796777 12282 external_mini_cluster.cc:1428] Setting key 4bc81a1e4bf2ae3a1e022a31ffa1271c
I20260812 21:04:26.797637 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786568607354921-12282-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786568607354921-12282-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786568607354921-12282-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786568607354921-12282-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.254.131:0
--local_ip_for_outbound_sockets=127.11.254.131
--webserver_interface=127.11.254.131
--webserver_port=0
--tserver_master_addrs=127.11.254.190:41747
--builtin_ntp_servers=127.11.254.148:37843
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260812 21:04:26.882323 29775 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:04:26.882591 29775 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:04:26.882619 29775 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:04:26.883863 29775 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:04:26.883903 29775 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:04:26.883929 29775 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.254.131
I20260812 21:04:26.885360 29775 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.254.148:37843
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786568607354921-12282-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786568607354921-12282-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.254.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786568607354921-12282-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.254.131
--webserver_port=0
--tserver_master_addrs=127.11.254.190:41747
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29775
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.254.131
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786568607354921-12282-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:04:26.885598 29775 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:04:26.885864 29775 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 21:04:26.891016 29775 server_base.cc:1034] running on GCE node
W20260812 21:04:26.890987 29783 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:04:26.890987 29789 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:04:26.891004 29787 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:04:26.891418 29775 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:04:26.891644 29775 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:04:26.892776 29775 hybrid_clock.cc:648] HybridClock initialized: now 1786568666892767 us; error 27 us; skew 500 ppm
I20260812 21:04:26.894100 29775 webserver.cc:460] Webserver started at http://127.11.254.131:42005/ using document root <none> and password file <none>
I20260812 21:04:26.894347 29775 fs_manager.cc:359] Metadata directory not provided
I20260812 21:04:26.894397 29775 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:04:26.894567 29775 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:04:26.895663 29775 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786568607354921-12282-0/minicluster-data/ts-2/data/instance:
uuid: "3e5f00eb21084b0aba70a9357001e05f"
format_stamp: "Formatted at 2026-08-12 21:04:26 on dist-test-slave-vpvm"
server_key: "2e750c7d97768e0beca50e3f1c2c27ea"
server_key_iv: "0680a6c56927040742066d13f905a70b"
server_key_version: "encryptionkey@0"
I20260812 21:04:26.895996 29775 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786568607354921-12282-0/minicluster-data/ts-2/wal/instance:
uuid: "3e5f00eb21084b0aba70a9357001e05f"
format_stamp: "Formatted at 2026-08-12 21:04:26 on dist-test-slave-vpvm"
server_key: "2e750c7d97768e0beca50e3f1c2c27ea"
server_key_iv: "0680a6c56927040742066d13f905a70b"
server_key_version: "encryptionkey@0"
I20260812 21:04:26.897274 29775 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 21:04:26.897964 29799 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:04:26.898106 29775 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 21:04:26.898192 29775 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786568607354921-12282-0/minicluster-data/ts-2/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786568607354921-12282-0/minicluster-data/ts-2/wal
uuid: "3e5f00eb21084b0aba70a9357001e05f"
format_stamp: "Formatted at 2026-08-12 21:04:26 on dist-test-slave-vpvm"
server_key: "2e750c7d97768e0beca50e3f1c2c27ea"
server_key_iv: "0680a6c56927040742066d13f905a70b"
server_key_version: "encryptionkey@0"
I20260812 21:04:26.898257 29775 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786568607354921-12282-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786568607354921-12282-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786568607354921-12282-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:04:26.901599 29775 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:04:26.901844 29775 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:04:26.901986 29775 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:04:26.902181 29775 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:04:26.902468 29775 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:04:26.902511 29775 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:04:26.902539 29775 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:04:26.902563 29775 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:04:26.908140 29775 rpc_server.cc:307] RPC server started. Bound to: 127.11.254.131:35217
I20260812 21:04:26.908179 29985 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.254.131:35217 every 8 connection(s)
I20260812 21:04:26.908484 29775 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786568607354921-12282-0/minicluster-data/ts-2/data/info.pb
I20260812 21:04:26.911324 29315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:26.909184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.131:51903 (local address 127.11.254.190:41747)
0812 21:04:26.909239 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0812 21:04:26.909241 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:04:26.909484 (+ 243us) server_negotiation.cc:407] Connection header received
0812 21:04:26.909944 (+ 460us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:04:26.909945 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:04:26.909980 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:04:26.910047 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:04:26.911144 (+ 1097us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:04:26.911145 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:04:26.911145 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 21:04:26.911148 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:04:26.911160 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 21:04:26.911164 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:04:26.911203 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0812 21:04:26.911256 (+ 53us) server_negotiation.cc:299] Negotiation successful
0812 21:04:26.911268 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260812 21:04:26.911506 29989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:26.909160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:41747 (local address 127.11.254.131:51903)
0812 21:04:26.909338 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0812 21:04:26.909378 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 21:04:26.909834 (+ 456us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:04:26.910115 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:04:26.910119 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:04:26.910128 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:04:26.911009 (+ 881us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 21:04:26.911009 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:04:26.911039 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 21:04:26.911068 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:04:26.911203 (+ 135us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:04:26.911206 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:04:26.911208 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 21:04:26.911257 (+ 49us) client_negotiation.cc:240] Negotiation successful
0812 21:04:26.911304 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":68,"threads_started":1}
I20260812 21:04:26.912006 29987 heartbeater.cc:347] Connected to a master server at 127.11.254.190:41747
I20260812 21:04:26.912091 29987 heartbeater.cc:464] Registering TS with master...
I20260812 21:04:26.912236 29987 heartbeater.cc:511] Master 127.11.254.190:41747 requested a full tablet report, sending...
I20260812 21:04:26.912588 29257 ts_manager.cc:194] Registered new tserver with Master: 3e5f00eb21084b0aba70a9357001e05f (127.11.254.131:35217)
I20260812 21:04:26.913092 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 29775
I20260812 21:04:26.913177 12282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786568607354921-12282-0/minicluster-data/ts-2/wal/instance
I20260812 21:04:26.913327 12282 external_mini_cluster.cc:1428] Setting key 045f2657bd5ca421c68f241536060dc0
I20260812 21:04:26.914084 12282 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260812 21:04:26.915845 29315 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:04:26.915355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44134 (local address 127.11.254.190:41747)
0812 21:04:26.915396 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0812 21:04:26.915398 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:04:26.915505 (+ 107us) server_negotiation.cc:407] Connection header received
0812 21:04:26.915567 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:04:26.915568 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:04:26.915573 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:04:26.915618 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:04:26.915777 (+ 159us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44134: BlockingRecv error: recv got EOF from 127.0.0.1:44134 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
I20260812 21:04:26.915912 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 29352
I20260812 21:04:26.934474 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 29561
I20260812 21:04:26.949396 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 29775
I20260812 21:04:26.955454 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 29207
2026-08-12T21:04:26Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (579 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568607354921-12282-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 21:04:27 dist-test-slave-vpvm krb5kdc[30002](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 21:04:27 dist-test-slave-vpvm krb5kdc[30002](info): set up 2 sockets
Aug 12 21:04:27 dist-test-slave-vpvm krb5kdc[30002](info): commencing operation
krb5kdc: starting...
W20260812 21:04:28.151772 12282 mini_kdc.cc:121] Time spent starting KDC: real 1.165s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568607354921-12282-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568607354921-12282-0/krb5kdc/test-user.keytab.
Aug 12 21:04:28 dist-test-slave-vpvm krb5kdc[30002](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568668, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T21:04:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T21:04:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.254.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.254.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.254.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.190.keytab.
Entry for principal kudu/127.11.254.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.190.keytab.
I20260812 21:04:28.281059 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568607354921-12282-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568607354921-12282-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568607354921-12282-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.254.190:40275
--webserver_interface=127.11.254.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.190.keytab
--principal=kudu/127.11.254.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.254.148:34629
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.254.190:40275
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568607354921-12282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568607354921-12282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568607354921-12282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:04:28.364718 30023 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:04:28.364941 30023 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:04:28.364974 30023 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:04:28.366160 30023 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 21:04:28.366204 30023 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:04:28.366223 30023 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:04:28.366240 30023 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 21:04:28.366255 30023 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 21:04:28.367790 30023 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.254.148:34629
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568607354921-12282-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568607354921-12282-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.254.190:40275
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.190.keytab
--principal=kudu/127.11.254.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.254.190:40275
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.254.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30023
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568607354921-12282-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:04:28.368053 30023 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:04:28.368274 30023 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:04:28.373091 30032 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:04:28.373096 30029 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:04:28.373107 30030 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:04:28.373402 30023 server_base.cc:1034] running on GCE node
I20260812 21:04:28.373667 30023 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:04:28.373919 30023 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:04:28.375083 30023 hybrid_clock.cc:648] HybridClock initialized: now 1786568668375068 us; error 26 us; skew 500 ppm
Aug 12 21:04:28 dist-test-slave-vpvm krb5kdc[30002](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568668, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:04:28.377693 30023 init.cc:377] Logged in from keytab as kudu/127.11.254.190@KRBTEST.COM (short username kudu)
I20260812 21:04:28.378183 30023 webserver.cc:460] Webserver started at http://127.11.254.190:35407/ using document root <none> and password file <none>
I20260812 21:04:28.378412 30023 fs_manager.cc:359] Metadata directory not provided
I20260812 21:04:28.378458 30023 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:04:28.378575 30023 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:04:28.379658 30023 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568607354921-12282-0/minicluster-data/master-0/data/instance:
uuid: "1f98b9482d3b48da9f25fea1e3019bf3"
format_stamp: "Formatted at 2026-08-12 21:04:28 on dist-test-slave-vpvm"
server_key: "d3478ecaefcef59eec1d3ad3e69faf2f"
server_key_iv: "07729018bdc7da3948106aeb4f846ca4"
server_key_version: "encryptionkey@0"
I20260812 21:04:28.379990 30023 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568607354921-12282-0/minicluster-data/master-0/wal/instance:
uuid: "1f98b9482d3b48da9f25fea1e3019bf3"
format_stamp: "Formatted at 2026-08-12 21:04:28 on dist-test-slave-vpvm"
server_key: "d3478ecaefcef59eec1d3ad3e69faf2f"
server_key_iv: "07729018bdc7da3948106aeb4f846ca4"
server_key_version: "encryptionkey@0"
I20260812 21:04:28.381239 30023 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 21:04:28.382036 30044 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:04:28.382206 30023 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 21:04:28.382287 30023 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568607354921-12282-0/minicluster-data/master-0/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568607354921-12282-0/minicluster-data/master-0/wal
uuid: "1f98b9482d3b48da9f25fea1e3019bf3"
format_stamp: "Formatted at 2026-08-12 21:04:28 on dist-test-slave-vpvm"
server_key: "d3478ecaefcef59eec1d3ad3e69faf2f"
server_key_iv: "07729018bdc7da3948106aeb4f846ca4"
server_key_version: "encryptionkey@0"
I20260812 21:04:28.382342 30023 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568607354921-12282-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568607354921-12282-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568607354921-12282-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:04:28.403204 30023 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:04:28.405274 30023 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:04:28.405393 30023 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:04:28.409396 30023 rpc_server.cc:307] RPC server started. Bound to: 127.11.254.190:40275
I20260812 21:04:28.409435 30130 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.254.190:40275 every 8 connection(s)
I20260812 21:04:28.409765 30023 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
I20260812 21:04:28.410769 30133 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:04:28.413156 30133 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f98b9482d3b48da9f25fea1e3019bf3: Bootstrap starting.
I20260812 21:04:28.413733 30133 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1f98b9482d3b48da9f25fea1e3019bf3: Neither blocks nor log segments found. Creating new log.
I20260812 21:04:28.414053 30133 log.cc:824] T 00000000000000000000000000000000 P 1f98b9482d3b48da9f25fea1e3019bf3: Log is configured to *not* fsync() on all Append() calls
I20260812 21:04:28.414749 30133 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f98b9482d3b48da9f25fea1e3019bf3: No bootstrap required, opened a new log
I20260812 21:04:28.416380 30133 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1f98b9482d3b48da9f25fea1e3019bf3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f98b9482d3b48da9f25fea1e3019bf3" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 40275 } }
I20260812 21:04:28.416528 30133 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1f98b9482d3b48da9f25fea1e3019bf3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:04:28.416563 30133 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1f98b9482d3b48da9f25fea1e3019bf3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f98b9482d3b48da9f25fea1e3019bf3, State: Initialized, Role: FOLLOWER
I20260812 21:04:28.416705 30133 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1f98b9482d3b48da9f25fea1e3019bf3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f98b9482d3b48da9f25fea1e3019bf3" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 40275 } }
I20260812 21:04:28.416786 30133 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1f98b9482d3b48da9f25fea1e3019bf3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 21:04:28.416817 30133 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1f98b9482d3b48da9f25fea1e3019bf3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 21:04:28.416867 30133 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1f98b9482d3b48da9f25fea1e3019bf3 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:04:28.417127 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 30023
I20260812 21:04:28.417208 12282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568607354921-12282-0/minicluster-data/master-0/wal/instance
I20260812 21:04:28.417372 12282 external_mini_cluster.cc:1428] Setting key f96da4e0c5e4dfb4c63710f9ccb58505
I20260812 21:04:28.417865 30133 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1f98b9482d3b48da9f25fea1e3019bf3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f98b9482d3b48da9f25fea1e3019bf3" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 40275 } }
I20260812 21:04:28.418026 30133 leader_election.cc:302] T 00000000000000000000000000000000 P 1f98b9482d3b48da9f25fea1e3019bf3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1f98b9482d3b48da9f25fea1e3019bf3; no voters:
I20260812 21:04:28.418211 30133 leader_election.cc:288] T 00000000000000000000000000000000 P 1f98b9482d3b48da9f25fea1e3019bf3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 21:04:28.418239 30149 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1f98b9482d3b48da9f25fea1e3019bf3 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:04:28.418359 30149 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1f98b9482d3b48da9f25fea1e3019bf3 [term 1 LEADER]: Becoming Leader. State: Replica: 1f98b9482d3b48da9f25fea1e3019bf3, State: Running, Role: LEADER
I20260812 21:04:28.418459 30133 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1f98b9482d3b48da9f25fea1e3019bf3 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 21:04:28.418453 30149 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1f98b9482d3b48da9f25fea1e3019bf3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f98b9482d3b48da9f25fea1e3019bf3" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 40275 } }
I20260812 21:04:28.418933 30153 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f98b9482d3b48da9f25fea1e3019bf3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1f98b9482d3b48da9f25fea1e3019bf3. Latest consensus state: current_term: 1 leader_uuid: "1f98b9482d3b48da9f25fea1e3019bf3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f98b9482d3b48da9f25fea1e3019bf3" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 40275 } } }
I20260812 21:04:28.419013 30153 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f98b9482d3b48da9f25fea1e3019bf3 [sys.catalog]: This master's current role is: LEADER
I20260812 21:04:28.419140 30154 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f98b9482d3b48da9f25fea1e3019bf3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1f98b9482d3b48da9f25fea1e3019bf3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f98b9482d3b48da9f25fea1e3019bf3" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 40275 } } }
I20260812 21:04:28.419195 30154 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f98b9482d3b48da9f25fea1e3019bf3 [sys.catalog]: This master's current role is: LEADER
I20260812 21:04:28.419620 30160 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 21:04:28.420145 30160 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 21:04:28.421780 30160 catalog_manager.cc:1347] Generated new cluster ID: 48619910dc94479f9d383b58736def15
I20260812 21:04:28.421834 30160 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 21:04:28 dist-test-slave-vpvm krb5kdc[30002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568668, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.254.190@KRBTEST.COM
I20260812 21:04:28.427589 30146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:28.417853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45302 (local address 127.11.254.190:40275)
0812 21:04:28.418099 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0812 21:04:28.418109 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 21:04:28.418145 (+ 36us) server_negotiation.cc:407] Connection header received
0812 21:04:28.418270 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:04:28.418278 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:04:28.418340 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:04:28.419257 (+ 917us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:04:28.419362 (+ 105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:28.420792 (+ 1430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:28.421762 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:28.422301 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:28.422403 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:28.424600 (+ 2197us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:04:28.424604 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:04:28.424605 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:04:28.424617 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:04:28.426232 (+ 1615us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:04:28.426543 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:04:28.426547 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:04:28.426547 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:04:28.426645 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:04:28.426822 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:04:28.426823 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:04:28.426823 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:04:28.427057 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:04:28.427230 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0812 21:04:28.427297 (+ 67us) server_negotiation.cc:299] Negotiation successful
0812 21:04:28.427431 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":192,"thread_start_us":121,"threads_started":1}
I20260812 21:04:28.427937 30160 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 21:04:28.428421 30160 catalog_manager.cc:1504] Loading token signing keys...
I20260812 21:04:28.434471 30160 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1f98b9482d3b48da9f25fea1e3019bf3: Generated new TSK 0
I20260812 21:04:28.434661 30160 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.254.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.254.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.254.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.129.keytab.
Entry for principal kudu/127.11.254.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.129.keytab.
I20260812 21:04:28.461071 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568607354921-12282-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568607354921-12282-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568607354921-12282-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568607354921-12282-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.254.129:0
--local_ip_for_outbound_sockets=127.11.254.129
--webserver_interface=127.11.254.129
--webserver_port=0
--tserver_master_addrs=127.11.254.190:40275
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.129.keytab
--principal=kudu/127.11.254.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.254.148:34629
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568607354921-12282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568607354921-12282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568607354921-12282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:04:28.544062 30180 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:04:28.544286 30180 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:04:28.544309 30180 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:04:28.545486 30180 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:04:28.545537 30180 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:04:28.545564 30180 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.254.129
I20260812 21:04:28.547138 30180 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.254.148:34629
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568607354921-12282-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568607354921-12282-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.129.keytab
--principal=kudu/127.11.254.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.254.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568607354921-12282-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.254.129
--webserver_port=0
--tserver_master_addrs=127.11.254.190:40275
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30180
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.254.129
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568607354921-12282-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:04:28.547374 30180 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:04:28.547605 30180 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 21:04:28.552862 30180 server_base.cc:1034] running on GCE node
W20260812 21:04:28.552798 30196 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:04:28.552799 30189 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:04:28.552793 30193 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:04:28.553205 30180 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:04:28.553464 30180 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:04:28.554616 30180 hybrid_clock.cc:648] HybridClock initialized: now 1786568668554601 us; error 29 us; skew 500 ppm
Aug 12 21:04:28 dist-test-slave-vpvm krb5kdc[30002](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568668, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:04:28.557108 30180 init.cc:377] Logged in from keytab as kudu/127.11.254.129@KRBTEST.COM (short username kudu)
I20260812 21:04:28.557534 30180 webserver.cc:460] Webserver started at http://127.11.254.129:42093/ using document root <none> and password file <none>
I20260812 21:04:28.557780 30180 fs_manager.cc:359] Metadata directory not provided
I20260812 21:04:28.557829 30180 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:04:28.557983 30180 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:04:28.559100 30180 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568607354921-12282-0/minicluster-data/ts-0/data/instance:
uuid: "8287c2babb474575be99f0623283c4c1"
format_stamp: "Formatted at 2026-08-12 21:04:28 on dist-test-slave-vpvm"
server_key: "cffe0f6ae2054f468211d5cc9abbef66"
server_key_iv: "301aef9a3aaeb8fd418af2a95807d482"
server_key_version: "encryptionkey@0"
I20260812 21:04:28.559475 30180 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568607354921-12282-0/minicluster-data/ts-0/wal/instance:
uuid: "8287c2babb474575be99f0623283c4c1"
format_stamp: "Formatted at 2026-08-12 21:04:28 on dist-test-slave-vpvm"
server_key: "cffe0f6ae2054f468211d5cc9abbef66"
server_key_iv: "301aef9a3aaeb8fd418af2a95807d482"
server_key_version: "encryptionkey@0"
I20260812 21:04:28.560739 30180 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 21:04:28.561400 30207 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:04:28.561546 30180 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 21:04:28.561630 30180 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568607354921-12282-0/minicluster-data/ts-0/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568607354921-12282-0/minicluster-data/ts-0/wal
uuid: "8287c2babb474575be99f0623283c4c1"
format_stamp: "Formatted at 2026-08-12 21:04:28 on dist-test-slave-vpvm"
server_key: "cffe0f6ae2054f468211d5cc9abbef66"
server_key_iv: "301aef9a3aaeb8fd418af2a95807d482"
server_key_version: "encryptionkey@0"
I20260812 21:04:28.561687 30180 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568607354921-12282-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568607354921-12282-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568607354921-12282-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:04:28.568703 30180 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:04:28.570655 30180 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:04:28.570794 30180 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:04:28.571031 30180 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:04:28.571369 30180 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:04:28.571404 30180 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:04:28.571434 30180 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:04:28.571458 30180 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:04:28.577279 30180 rpc_server.cc:307] RPC server started. Bound to: 127.11.254.129:42693
I20260812 21:04:28.577335 30389 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.254.129:42693 every 8 connection(s)
I20260812 21:04:28.577664 30180 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568607354921-12282-0/minicluster-data/ts-0/data/info.pb
Aug 12 21:04:28 dist-test-slave-vpvm krb5kdc[30002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568668, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.129@KRBTEST.COM for kudu/127.11.254.190@KRBTEST.COM
I20260812 21:04:28.586589 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 30180
I20260812 21:04:28.586668 30146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:28.578443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.129:33945 (local address 127.11.254.190:40275)
0812 21:04:28.578507 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0812 21:04:28.578509 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:04:28.578683 (+ 174us) server_negotiation.cc:407] Connection header received
0812 21:04:28.579226 (+ 543us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:04:28.579227 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:04:28.579243 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:04:28.579292 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:04:28.580335 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:28.580857 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:28.581855 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:28.582122 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:28.582182 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:28.584230 (+ 2048us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:04:28.584231 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:04:28.584232 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:04:28.584241 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:04:28.585445 (+ 1204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:04:28.585801 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:04:28.585801 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:04:28.585802 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:04:28.585857 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:04:28.586082 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:04:28.586082 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:04:28.586082 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:04:28.586206 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:04:28.586256 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0812 21:04:28.586505 (+ 249us) server_negotiation.cc:299] Negotiation successful
0812 21:04:28.586598 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260812 21:04:28.586704 12282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568607354921-12282-0/minicluster-data/ts-0/wal/instance
I20260812 21:04:28.586889 12282 external_mini_cluster.cc:1428] Setting key e5d42540c82f656ca83bffe6b091c54c
I20260812 21:04:28.586894 30393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:28.578382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:40275 (local address 127.11.254.129:33945)
0812 21:04:28.578558 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0812 21:04:28.578599 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 21:04:28.579116 (+ 517us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:04:28.579376 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:04:28.579381 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:04:28.580027 (+ 646us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:04:28.580228 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:28.580233 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:28.580943 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:28.580944 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:28.581765 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:28.581767 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:28.582198 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:28.582199 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:28.582319 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:28.582751 (+ 432us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:04:28.582759 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:04:28.584136 (+ 1377us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:04:28.585507 (+ 1371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:04:28.585513 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:04:28.585514 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:04:28.585731 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:04:28.585905 (+ 174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:04:28.585906 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:04:28.585906 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:04:28.586033 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:04:28.586251 (+ 218us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:04:28.586257 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:04:28.586379 (+ 122us) client_negotiation.cc:769] Sending connection context
0812 21:04:28.586495 (+ 116us) client_negotiation.cc:240] Negotiation successful
0812 21:04:28.586667 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":94,"threads_started":1}
I20260812 21:04:28.587368 30390 heartbeater.cc:347] Connected to a master server at 127.11.254.190:40275
I20260812 21:04:28.587463 30390 heartbeater.cc:464] Registering TS with master...
I20260812 21:04:28.587659 30390 heartbeater.cc:511] Master 127.11.254.190:40275 requested a full tablet report, sending...
I20260812 21:04:28.588238 30074 ts_manager.cc:194] Registered new tserver with Master: 8287c2babb474575be99f0623283c4c1 (127.11.254.129:42693)
I20260812 21:04:28.589001 30074 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.254.129@KRBTEST.COM'} at 127.11.254.129:33945
WARNING: no policy specified for kudu/127.11.254.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.254.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.254.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.130.keytab.
Entry for principal kudu/127.11.254.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.130.keytab.
I20260812 21:04:28.614415 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568607354921-12282-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568607354921-12282-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568607354921-12282-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568607354921-12282-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.254.130:0
--local_ip_for_outbound_sockets=127.11.254.130
--webserver_interface=127.11.254.130
--webserver_port=0
--tserver_master_addrs=127.11.254.190:40275
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.130.keytab
--principal=kudu/127.11.254.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.254.148:34629
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568607354921-12282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568607354921-12282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568607354921-12282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:04:28.698556 30397 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:04:28.698781 30397 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:04:28.698804 30397 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:04:28.700016 30397 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:04:28.700064 30397 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:04:28.700091 30397 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.254.130
I20260812 21:04:28.701610 30397 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.254.148:34629
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568607354921-12282-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568607354921-12282-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.130.keytab
--principal=kudu/127.11.254.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.254.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568607354921-12282-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.254.130
--webserver_port=0
--tserver_master_addrs=127.11.254.190:40275
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30397
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.254.130
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568607354921-12282-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:04:28.701858 30397 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:04:28.702253 30397 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:04:28.707486 30416 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:04:28.707472 30413 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:04:28.707656 30397 server_base.cc:1034] running on GCE node
W20260812 21:04:28.707476 30409 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:04:28.707911 30397 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:04:28.708138 30397 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:04:28.709297 30397 hybrid_clock.cc:648] HybridClock initialized: now 1786568668709278 us; error 37 us; skew 500 ppm
Aug 12 21:04:28 dist-test-slave-vpvm krb5kdc[30002](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568668, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:04:28.711918 30397 init.cc:377] Logged in from keytab as kudu/127.11.254.130@KRBTEST.COM (short username kudu)
I20260812 21:04:28.712452 30397 webserver.cc:460] Webserver started at http://127.11.254.130:42047/ using document root <none> and password file <none>
I20260812 21:04:28.712685 30397 fs_manager.cc:359] Metadata directory not provided
I20260812 21:04:28.712735 30397 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:04:28.712863 30397 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:04:28.713919 30397 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568607354921-12282-0/minicluster-data/ts-1/data/instance:
uuid: "e19c8a9b7ad74aca89883d87023b5c0b"
format_stamp: "Formatted at 2026-08-12 21:04:28 on dist-test-slave-vpvm"
server_key: "0e7253e2df243bcea9888c65f506bb85"
server_key_iv: "1d2dc4c25222a999be1f7a63598e463c"
server_key_version: "encryptionkey@0"
I20260812 21:04:28.714310 30397 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568607354921-12282-0/minicluster-data/ts-1/wal/instance:
uuid: "e19c8a9b7ad74aca89883d87023b5c0b"
format_stamp: "Formatted at 2026-08-12 21:04:28 on dist-test-slave-vpvm"
server_key: "0e7253e2df243bcea9888c65f506bb85"
server_key_iv: "1d2dc4c25222a999be1f7a63598e463c"
server_key_version: "encryptionkey@0"
I20260812 21:04:28.715626 30397 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 21:04:28.716372 30427 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:04:28.716580 30397 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 21:04:28.716681 30397 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568607354921-12282-0/minicluster-data/ts-1/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568607354921-12282-0/minicluster-data/ts-1/wal
uuid: "e19c8a9b7ad74aca89883d87023b5c0b"
format_stamp: "Formatted at 2026-08-12 21:04:28 on dist-test-slave-vpvm"
server_key: "0e7253e2df243bcea9888c65f506bb85"
server_key_iv: "1d2dc4c25222a999be1f7a63598e463c"
server_key_version: "encryptionkey@0"
I20260812 21:04:28.716753 30397 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568607354921-12282-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568607354921-12282-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568607354921-12282-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:04:28.731007 30397 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:04:28.732859 30397 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:04:28.733013 30397 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:04:28.733240 30397 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:04:28.733552 30397 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:04:28.733587 30397 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:04:28.733618 30397 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:04:28.733641 30397 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:04:28.739375 30397 rpc_server.cc:307] RPC server started. Bound to: 127.11.254.130:41989
I20260812 21:04:28.739439 30600 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.254.130:41989 every 8 connection(s)
I20260812 21:04:28.739751 30397 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568607354921-12282-0/minicluster-data/ts-1/data/info.pb
I20260812 21:04:28.740242 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 30397
I20260812 21:04:28.740337 12282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568607354921-12282-0/minicluster-data/ts-1/wal/instance
I20260812 21:04:28.740628 12282 external_mini_cluster.cc:1428] Setting key 245879c8f50e11e483a2a64fdf2c91af
Aug 12 21:04:28 dist-test-slave-vpvm krb5kdc[30002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568668, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.130@KRBTEST.COM for kudu/127.11.254.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.11.254.131@KRBTEST.COM; defaulting to no policy
I20260812 21:04:28.750162 30146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:28.740483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.130:51037 (local address 127.11.254.190:40275)
0812 21:04:28.740551 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0812 21:04:28.740554 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 21:04:28.740737 (+ 183us) server_negotiation.cc:407] Connection header received
0812 21:04:28.741401 (+ 664us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:04:28.741403 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:04:28.741416 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:04:28.741512 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:04:28.742626 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:28.743331 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:28.744802 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:28.744990 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:28.745037 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:28.747464 (+ 2427us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:04:28.747465 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:04:28.747466 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:04:28.747473 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:04:28.748717 (+ 1244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:04:28.749136 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:04:28.749137 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:04:28.749138 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:04:28.749203 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:04:28.749452 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:04:28.749452 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:04:28.749453 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:04:28.749600 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:04:28.749674 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0812 21:04:28.749935 (+ 261us) server_negotiation.cc:299] Negotiation successful
0812 21:04:28.750056 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260812 21:04:28.750285 30605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:28.740452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:40275 (local address 127.11.254.130:51037)
0812 21:04:28.740612 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0812 21:04:28.740655 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 21:04:28.741274 (+ 619us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:04:28.741632 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:04:28.741640 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:04:28.742282 (+ 642us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:04:28.742484 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:28.742489 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:28.743446 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:28.743447 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:28.744723 (+ 1276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:28.744726 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:28.745064 (+ 338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:28.745065 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:28.745222 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:28.745756 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:04:28.745766 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:04:28.747367 (+ 1601us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:04:28.748795 (+ 1428us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:04:28.748802 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:04:28.748805 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:04:28.749042 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:04:28.749281 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:04:28.749282 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:04:28.749282 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:04:28.749387 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:04:28.749662 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:04:28.749666 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:04:28.749791 (+ 125us) client_negotiation.cc:769] Sending connection context
0812 21:04:28.749895 (+ 104us) client_negotiation.cc:240] Negotiation successful
0812 21:04:28.750078 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":86,"threads_started":1}
I20260812 21:04:28.750802 30602 heartbeater.cc:347] Connected to a master server at 127.11.254.190:40275
I20260812 21:04:28.750909 30602 heartbeater.cc:464] Registering TS with master...
I20260812 21:04:28.751091 30602 heartbeater.cc:511] Master 127.11.254.190:40275 requested a full tablet report, sending...
I20260812 21:04:28.751478 30074 ts_manager.cc:194] Registered new tserver with Master: e19c8a9b7ad74aca89883d87023b5c0b (127.11.254.130:41989)
I20260812 21:04:28.751880 30074 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.254.130@KRBTEST.COM'} at 127.11.254.130:51037
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.254.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.254.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.131.keytab.
Entry for principal kudu/127.11.254.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.131.keytab.
I20260812 21:04:28.769284 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568607354921-12282-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568607354921-12282-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568607354921-12282-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568607354921-12282-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.254.131:0
--local_ip_for_outbound_sockets=127.11.254.131
--webserver_interface=127.11.254.131
--webserver_port=0
--tserver_master_addrs=127.11.254.190:40275
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.131.keytab
--principal=kudu/127.11.254.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.254.148:34629
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568607354921-12282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568607354921-12282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568607354921-12282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:04:28.853235 30609 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:04:28.853466 30609 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:04:28.853497 30609 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:04:28.854692 30609 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:04:28.854737 30609 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:04:28.854771 30609 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.254.131
I20260812 21:04:28.856259 30609 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.254.148:34629
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568607354921-12282-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568607354921-12282-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.131.keytab
--principal=kudu/127.11.254.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.254.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568607354921-12282-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.254.131
--webserver_port=0
--tserver_master_addrs=127.11.254.190:40275
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30609
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.254.131
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568607354921-12282-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:04:28.856530 30609 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:04:28.856750 30609 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:04:28.862051 30625 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:04:28.862135 30609 server_base.cc:1034] running on GCE node
W20260812 21:04:28.862077 30619 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:04:28.862046 30618 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:04:28.862586 30609 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:04:28.862802 30609 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:04:28.863961 30609 hybrid_clock.cc:648] HybridClock initialized: now 1786568668863935 us; error 39 us; skew 500 ppm
Aug 12 21:04:28 dist-test-slave-vpvm krb5kdc[30002](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568668, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:04:28.866653 30609 init.cc:377] Logged in from keytab as kudu/127.11.254.131@KRBTEST.COM (short username kudu)
I20260812 21:04:28.867142 30609 webserver.cc:460] Webserver started at http://127.11.254.131:43955/ using document root <none> and password file <none>
I20260812 21:04:28.867349 30609 fs_manager.cc:359] Metadata directory not provided
I20260812 21:04:28.867393 30609 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:04:28.867533 30609 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:04:28.868642 30609 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568607354921-12282-0/minicluster-data/ts-2/data/instance:
uuid: "75433f0931d34bed963cb1cca3a74146"
format_stamp: "Formatted at 2026-08-12 21:04:28 on dist-test-slave-vpvm"
server_key: "a846b2646a5dc81c2358b743e9ff2a27"
server_key_iv: "bc27be4a57e6dd7766a8892dd223fedb"
server_key_version: "encryptionkey@0"
I20260812 21:04:28.868996 30609 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568607354921-12282-0/minicluster-data/ts-2/wal/instance:
uuid: "75433f0931d34bed963cb1cca3a74146"
format_stamp: "Formatted at 2026-08-12 21:04:28 on dist-test-slave-vpvm"
server_key: "a846b2646a5dc81c2358b743e9ff2a27"
server_key_iv: "bc27be4a57e6dd7766a8892dd223fedb"
server_key_version: "encryptionkey@0"
I20260812 21:04:28.870541 30609 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 21:04:28.871228 30635 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:04:28.871377 30609 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 21:04:28.871517 30609 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568607354921-12282-0/minicluster-data/ts-2/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568607354921-12282-0/minicluster-data/ts-2/wal
uuid: "75433f0931d34bed963cb1cca3a74146"
format_stamp: "Formatted at 2026-08-12 21:04:28 on dist-test-slave-vpvm"
server_key: "a846b2646a5dc81c2358b743e9ff2a27"
server_key_iv: "bc27be4a57e6dd7766a8892dd223fedb"
server_key_version: "encryptionkey@0"
I20260812 21:04:28.871587 30609 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568607354921-12282-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568607354921-12282-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568607354921-12282-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:04:28.888787 30609 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:04:28.890786 30609 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:04:28.890918 30609 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:04:28.891146 30609 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:04:28.891525 30609 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:04:28.891566 30609 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:04:28.891598 30609 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:04:28.891621 30609 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:04:28.897369 30609 rpc_server.cc:307] RPC server started. Bound to: 127.11.254.131:43887
I20260812 21:04:28.897413 30820 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.254.131:43887 every 8 connection(s)
I20260812 21:04:28.897854 30609 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568607354921-12282-0/minicluster-data/ts-2/data/info.pb
Aug 12 21:04:28 dist-test-slave-vpvm krb5kdc[30002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568668, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.131@KRBTEST.COM for kudu/127.11.254.190@KRBTEST.COM
I20260812 21:04:28.905143 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 30609
I20260812 21:04:28.905246 12282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568607354921-12282-0/minicluster-data/ts-2/wal/instance
I20260812 21:04:28.905447 12282 external_mini_cluster.cc:1428] Setting key 826c984e4077e23609729d69c3d5000d
I20260812 21:04:28.907807 30146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:28.899186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.131:49233 (local address 127.11.254.190:40275)
0812 21:04:28.899270 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0812 21:04:28.899272 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:04:28.899465 (+ 193us) server_negotiation.cc:407] Connection header received
0812 21:04:28.899944 (+ 479us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:04:28.899945 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:04:28.899960 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:04:28.900040 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:04:28.900978 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:28.901533 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:28.902628 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:28.902837 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:28.902903 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:28.905108 (+ 2205us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:04:28.905109 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:04:28.905110 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:04:28.905117 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:04:28.906643 (+ 1526us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:04:28.907035 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:04:28.907036 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:04:28.907036 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:04:28.907099 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:04:28.907292 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:04:28.907293 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:04:28.907293 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:04:28.907396 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:04:28.907495 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0812 21:04:28.907667 (+ 172us) server_negotiation.cc:299] Negotiation successful
0812 21:04:28.907740 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260812 21:04:28.907966 30826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:28.899125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:40275 (local address 127.11.254.131:49233)
0812 21:04:28.899290 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0812 21:04:28.899334 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 21:04:28.899830 (+ 496us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:04:28.900098 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:04:28.900100 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:04:28.900686 (+ 586us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:04:28.900885 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:28.900890 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:28.901629 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:28.901631 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:28.902520 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:28.902523 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:28.902912 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:28.902913 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:28.903022 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:28.903525 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:04:28.903533 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:04:28.904996 (+ 1463us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:04:28.906722 (+ 1726us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:04:28.906728 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:04:28.906730 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:04:28.906955 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:04:28.907152 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:04:28.907152 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:04:28.907153 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:04:28.907249 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:04:28.907437 (+ 188us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:04:28.907440 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:04:28.907562 (+ 122us) client_negotiation.cc:769] Sending connection context
0812 21:04:28.907644 (+ 82us) client_negotiation.cc:240] Negotiation successful
0812 21:04:28.907773 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":79,"threads_started":1}
I20260812 21:04:28.908408 30821 heartbeater.cc:347] Connected to a master server at 127.11.254.190:40275
I20260812 21:04:28.908499 30821 heartbeater.cc:464] Registering TS with master...
I20260812 21:04:28.908658 30821 heartbeater.cc:511] Master 127.11.254.190:40275 requested a full tablet report, sending...
I20260812 21:04:28.909016 30074 ts_manager.cc:194] Registered new tserver with Master: 75433f0931d34bed963cb1cca3a74146 (127.11.254.131:43887)
I20260812 21:04:28.909376 30074 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.254.131@KRBTEST.COM'} at 127.11.254.131:49233
I20260812 21:04:28.917460 12282 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 21:04:28.925148 30146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:28.918842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59018 (local address 127.11.254.190:40275)
0812 21:04:28.918897 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0812 21:04:28.918899 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:04:28.919039 (+ 140us) server_negotiation.cc:407] Connection header received
0812 21:04:28.919104 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:04:28.919105 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:04:28.919116 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:04:28.919164 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:04:28.919819 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:28.920371 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:28.921136 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:28.921330 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:28.921385 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:28.922544 (+ 1159us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:04:28.922545 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:04:28.922546 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:04:28.922553 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:04:28.923919 (+ 1366us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:04:28.924269 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:04:28.924270 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:04:28.924270 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:04:28.924334 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:04:28.924526 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:04:28.924527 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:04:28.924527 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:04:28.924698 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:04:28.924808 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0812 21:04:28.924951 (+ 143us) server_negotiation.cc:299] Negotiation successful
0812 21:04:28.925053 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 21:04:28.926692 30074 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59018:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 21:04:28.928313 30074 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 21:04:28.934686 30836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:28.930646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.131:43887 (local address 127.0.0.1:40576)
0812 21:04:28.930949 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0812 21:04:28.930960 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 21:04:28.931018 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:04:28.931294 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:04:28.931296 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:04:28.931298 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:04:28.931560 (+ 262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:28.931564 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:28.932747 (+ 1183us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:28.932748 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:28.933642 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:28.933644 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:28.934370 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:28.934371 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:28.934456 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:28.934470 (+ 14us) client_negotiation.cc:769] Sending connection context
0812 21:04:28.934549 (+ 79us) client_negotiation.cc:240] Negotiation successful
0812 21:04:28.934605 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":286,"thread_start_us":42,"threads_started":1}
I20260812 21:04:28.934702 30838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:28.930742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40576 (local address 127.11.254.131:43887)
0812 21:04:28.930940 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0812 21:04:28.930945 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 21:04:28.931079 (+ 134us) server_negotiation.cc:407] Connection header received
0812 21:04:28.931124 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:04:28.931126 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:04:28.931142 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:04:28.931212 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:04:28.931650 (+ 438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:28.932650 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:28.933722 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:28.934284 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:28.934362 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:28.934432 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 21:04:28.934564 (+ 132us) server_negotiation.cc:299] Negotiation successful
0812 21:04:28.934605 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":114,"threads_started":1}
I20260812 21:04:28.935024 30835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:28.930570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43284 (local address 127.11.254.129:42693)
0812 21:04:28.930798 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0812 21:04:28.930803 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 21:04:28.930816 (+ 13us) server_negotiation.cc:407] Connection header received
0812 21:04:28.930861 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:04:28.930864 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:04:28.930907 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:04:28.931017 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:04:28.931196 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:28.932279 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:28.933760 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:28.934526 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:28.934596 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:28.934648 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0812 21:04:28.934905 (+ 257us) server_negotiation.cc:299] Negotiation successful
0812 21:04:28.934942 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":194,"thread_start_us":112,"threads_started":1}
I20260812 21:04:28.935137 30839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:28.930902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45478 (local address 127.11.254.130:41989)
0812 21:04:28.931075 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0812 21:04:28.931079 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 21:04:28.931206 (+ 127us) server_negotiation.cc:407] Connection header received
0812 21:04:28.931319 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:04:28.931321 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:04:28.931336 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:04:28.931407 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:04:28.931504 (+ 97us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:28.932177 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:28.933760 (+ 1583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:28.934525 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:28.934596 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:28.934654 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0812 21:04:28.935016 (+ 362us) server_negotiation.cc:299] Negotiation successful
0812 21:04:28.935063 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":74,"threads_started":1}
I20260812 21:04:28.934943 30837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:28.930776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.130:41989 (local address 127.0.0.1:45478)
0812 21:04:28.931128 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0812 21:04:28.931138 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 21:04:28.931233 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:04:28.931409 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:04:28.931410 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:04:28.931412 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:04:28.931453 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:28.931455 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:28.932269 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:28.932270 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:28.933658 (+ 1388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:28.933661 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:28.934688 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:28.934689 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:28.934758 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:28.934764 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 21:04:28.934812 (+ 48us) client_negotiation.cc:240] Negotiation successful
0812 21:04:28.934862 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":329,"mutex_wait_us":36,"thread_start_us":44,"threads_started":1}
I20260812 21:04:28.935261 30834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:28.930426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.129:42693 (local address 127.0.0.1:43284)
0812 21:04:28.930540 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0812 21:04:28.930581 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 21:04:28.930642 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:04:28.931027 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:04:28.931030 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:04:28.931033 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:04:28.931113 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:28.931114 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:28.932380 (+ 1266us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:28.932381 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:28.933659 (+ 1278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:28.933661 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:28.934689 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:28.934690 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:28.934758 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:28.934765 (+ 7us) client_negotiation.cc:769] Sending connection context
0812 21:04:28.934812 (+ 47us) client_negotiation.cc:240] Negotiation successful
0812 21:04:28.934861 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":87,"mutex_wait_us":40,"thread_start_us":52,"threads_started":1}
I20260812 21:04:28.935997 30715 tablet_service.cc:1467] Processing CreateTablet for tablet 483cc9bbe4a34f44b907b711f75b6070 (DEFAULT_TABLE table=test-table [id=2dc0cadf1faf429aab1ed0b72984e35b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:04:28.935999 30496 tablet_service.cc:1467] Processing CreateTablet for tablet 483cc9bbe4a34f44b907b711f75b6070 (DEFAULT_TABLE table=test-table [id=2dc0cadf1faf429aab1ed0b72984e35b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:04:28.935999 30286 tablet_service.cc:1467] Processing CreateTablet for tablet 483cc9bbe4a34f44b907b711f75b6070 (DEFAULT_TABLE table=test-table [id=2dc0cadf1faf429aab1ed0b72984e35b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:04:28.936357 30286 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 483cc9bbe4a34f44b907b711f75b6070. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:04:28.936357 30496 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 483cc9bbe4a34f44b907b711f75b6070. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:04:28.936357 30715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 483cc9bbe4a34f44b907b711f75b6070. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:04:28.939322 30840 tablet_bootstrap.cc:492] T 483cc9bbe4a34f44b907b711f75b6070 P 75433f0931d34bed963cb1cca3a74146: Bootstrap starting.
I20260812 21:04:28.939738 30842 tablet_bootstrap.cc:492] T 483cc9bbe4a34f44b907b711f75b6070 P 8287c2babb474575be99f0623283c4c1: Bootstrap starting.
I20260812 21:04:28.939918 30841 tablet_bootstrap.cc:492] T 483cc9bbe4a34f44b907b711f75b6070 P e19c8a9b7ad74aca89883d87023b5c0b: Bootstrap starting.
I20260812 21:04:28.940189 30840 tablet_bootstrap.cc:654] T 483cc9bbe4a34f44b907b711f75b6070 P 75433f0931d34bed963cb1cca3a74146: Neither blocks nor log segments found. Creating new log.
I20260812 21:04:28.940414 30842 tablet_bootstrap.cc:654] T 483cc9bbe4a34f44b907b711f75b6070 P 8287c2babb474575be99f0623283c4c1: Neither blocks nor log segments found. Creating new log.
I20260812 21:04:28.940532 30841 tablet_bootstrap.cc:654] T 483cc9bbe4a34f44b907b711f75b6070 P e19c8a9b7ad74aca89883d87023b5c0b: Neither blocks nor log segments found. Creating new log.
I20260812 21:04:28.940770 30840 log.cc:824] T 483cc9bbe4a34f44b907b711f75b6070 P 75433f0931d34bed963cb1cca3a74146: Log is configured to *not* fsync() on all Append() calls
I20260812 21:04:28.940989 30842 log.cc:824] T 483cc9bbe4a34f44b907b711f75b6070 P 8287c2babb474575be99f0623283c4c1: Log is configured to *not* fsync() on all Append() calls
I20260812 21:04:28.941051 30841 log.cc:824] T 483cc9bbe4a34f44b907b711f75b6070 P e19c8a9b7ad74aca89883d87023b5c0b: Log is configured to *not* fsync() on all Append() calls
I20260812 21:04:28.941584 30840 tablet_bootstrap.cc:492] T 483cc9bbe4a34f44b907b711f75b6070 P 75433f0931d34bed963cb1cca3a74146: No bootstrap required, opened a new log
I20260812 21:04:28.941659 30840 ts_tablet_manager.cc:1397] T 483cc9bbe4a34f44b907b711f75b6070 P 75433f0931d34bed963cb1cca3a74146: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 21:04:28.941723 30841 tablet_bootstrap.cc:492] T 483cc9bbe4a34f44b907b711f75b6070 P e19c8a9b7ad74aca89883d87023b5c0b: No bootstrap required, opened a new log
I20260812 21:04:28.941783 30841 ts_tablet_manager.cc:1397] T 483cc9bbe4a34f44b907b711f75b6070 P e19c8a9b7ad74aca89883d87023b5c0b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 21:04:28.943282 30841 raft_consensus.cc:348] T 483cc9bbe4a34f44b907b711f75b6070 P e19c8a9b7ad74aca89883d87023b5c0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8287c2babb474575be99f0623283c4c1" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 42693 } } peers { permanent_uuid: "75433f0931d34bed963cb1cca3a74146" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 43887 } } peers { permanent_uuid: "e19c8a9b7ad74aca89883d87023b5c0b" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 41989 } }
I20260812 21:04:28.943430 30841 raft_consensus.cc:374] T 483cc9bbe4a34f44b907b711f75b6070 P e19c8a9b7ad74aca89883d87023b5c0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:04:28.943454 30841 raft_consensus.cc:729] T 483cc9bbe4a34f44b907b711f75b6070 P e19c8a9b7ad74aca89883d87023b5c0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e19c8a9b7ad74aca89883d87023b5c0b, State: Initialized, Role: FOLLOWER
I20260812 21:04:28.943400 30840 raft_consensus.cc:348] T 483cc9bbe4a34f44b907b711f75b6070 P 75433f0931d34bed963cb1cca3a74146 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8287c2babb474575be99f0623283c4c1" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 42693 } } peers { permanent_uuid: "75433f0931d34bed963cb1cca3a74146" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 43887 } } peers { permanent_uuid: "e19c8a9b7ad74aca89883d87023b5c0b" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 41989 } }
I20260812 21:04:28.943569 30840 raft_consensus.cc:374] T 483cc9bbe4a34f44b907b711f75b6070 P 75433f0931d34bed963cb1cca3a74146 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:04:28.943553 30841 consensus_queue.cc:260] T 483cc9bbe4a34f44b907b711f75b6070 P e19c8a9b7ad74aca89883d87023b5c0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8287c2babb474575be99f0623283c4c1" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 42693 } } peers { permanent_uuid: "75433f0931d34bed963cb1cca3a74146" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 43887 } } peers { permanent_uuid: "e19c8a9b7ad74aca89883d87023b5c0b" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 41989 } }
I20260812 21:04:28.943620 30840 raft_consensus.cc:729] T 483cc9bbe4a34f44b907b711f75b6070 P 75433f0931d34bed963cb1cca3a74146 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75433f0931d34bed963cb1cca3a74146, State: Initialized, Role: FOLLOWER
I20260812 21:04:28.943734 30840 consensus_queue.cc:260] T 483cc9bbe4a34f44b907b711f75b6070 P 75433f0931d34bed963cb1cca3a74146 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8287c2babb474575be99f0623283c4c1" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 42693 } } peers { permanent_uuid: "75433f0931d34bed963cb1cca3a74146" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 43887 } } peers { permanent_uuid: "e19c8a9b7ad74aca89883d87023b5c0b" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 41989 } }
I20260812 21:04:28.943826 30841 ts_tablet_manager.cc:1428] T 483cc9bbe4a34f44b907b711f75b6070 P e19c8a9b7ad74aca89883d87023b5c0b: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260812 21:04:28.943845 30602 heartbeater.cc:503] Master 127.11.254.190:40275 was elected leader, sending a full tablet report...
I20260812 21:04:28.944021 30840 ts_tablet_manager.cc:1428] T 483cc9bbe4a34f44b907b711f75b6070 P 75433f0931d34bed963cb1cca3a74146: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260812 21:04:28.944046 30821 heartbeater.cc:503] Master 127.11.254.190:40275 was elected leader, sending a full tablet report...
I20260812 21:04:28.944653 30842 tablet_bootstrap.cc:492] T 483cc9bbe4a34f44b907b711f75b6070 P 8287c2babb474575be99f0623283c4c1: No bootstrap required, opened a new log
I20260812 21:04:28.944730 30842 ts_tablet_manager.cc:1397] T 483cc9bbe4a34f44b907b711f75b6070 P 8287c2babb474575be99f0623283c4c1: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260812 21:04:28.946359 30842 raft_consensus.cc:348] T 483cc9bbe4a34f44b907b711f75b6070 P 8287c2babb474575be99f0623283c4c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8287c2babb474575be99f0623283c4c1" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 42693 } } peers { permanent_uuid: "75433f0931d34bed963cb1cca3a74146" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 43887 } } peers { permanent_uuid: "e19c8a9b7ad74aca89883d87023b5c0b" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 41989 } }
I20260812 21:04:28.946542 30842 raft_consensus.cc:374] T 483cc9bbe4a34f44b907b711f75b6070 P 8287c2babb474575be99f0623283c4c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:04:28.946590 30842 raft_consensus.cc:729] T 483cc9bbe4a34f44b907b711f75b6070 P 8287c2babb474575be99f0623283c4c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8287c2babb474575be99f0623283c4c1, State: Initialized, Role: FOLLOWER
I20260812 21:04:28.946717 30842 consensus_queue.cc:260] T 483cc9bbe4a34f44b907b711f75b6070 P 8287c2babb474575be99f0623283c4c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8287c2babb474575be99f0623283c4c1" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 42693 } } peers { permanent_uuid: "75433f0931d34bed963cb1cca3a74146" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 43887 } } peers { permanent_uuid: "e19c8a9b7ad74aca89883d87023b5c0b" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 41989 } }
I20260812 21:04:28.946993 30842 ts_tablet_manager.cc:1428] T 483cc9bbe4a34f44b907b711f75b6070 P 8287c2babb474575be99f0623283c4c1: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260812 21:04:28.947079 30390 heartbeater.cc:503] Master 127.11.254.190:40275 was elected leader, sending a full tablet report...
I20260812 21:04:28.984337 30848 raft_consensus.cc:482] T 483cc9bbe4a34f44b907b711f75b6070 P 75433f0931d34bed963cb1cca3a74146 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 21:04:28.984493 30848 raft_consensus.cc:504] T 483cc9bbe4a34f44b907b711f75b6070 P 75433f0931d34bed963cb1cca3a74146 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8287c2babb474575be99f0623283c4c1" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 42693 } } peers { permanent_uuid: "75433f0931d34bed963cb1cca3a74146" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 43887 } } peers { permanent_uuid: "e19c8a9b7ad74aca89883d87023b5c0b" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 41989 } }
I20260812 21:04:28.984978 30848 leader_election.cc:288] T 483cc9bbe4a34f44b907b711f75b6070 P 75433f0931d34bed963cb1cca3a74146 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8287c2babb474575be99f0623283c4c1 (127.11.254.129:42693), e19c8a9b7ad74aca89883d87023b5c0b (127.11.254.130:41989)
I20260812 21:04:28.988718 30826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:28.985157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.129:42693 (local address 127.11.254.131:60057)
0812 21:04:28.985216 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0812 21:04:28.985232 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 21:04:28.985297 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:04:28.985436 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:04:28.985437 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:04:28.985440 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:04:28.985523 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:28.985526 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:28.986483 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:28.986484 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:28.987684 (+ 1200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:28.987687 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:28.988461 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:28.988461 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:28.988548 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:28.988552 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 21:04:28.988598 (+ 46us) client_negotiation.cc:240] Negotiation successful
0812 21:04:28.988641 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260812 21:04:28.988725 30835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:28.985241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.131:60057 (local address 127.11.254.129:42693)
0812 21:04:28.985303 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0812 21:04:28.985314 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 21:04:28.985324 (+ 10us) server_negotiation.cc:407] Connection header received
0812 21:04:28.985354 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:04:28.985355 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:04:28.985364 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:04:28.985418 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:04:28.985593 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:28.986392 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:28.987766 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:28.988363 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:28.988440 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:28.988449 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 21:04:28.988623 (+ 174us) server_negotiation.cc:299] Negotiation successful
0812 21:04:28.988663 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260812 21:04:28.988864 30839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:28.985360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.131:55597 (local address 127.11.254.130:41989)
0812 21:04:28.985419 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0812 21:04:28.985422 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 21:04:28.985572 (+ 150us) server_negotiation.cc:407] Connection header received
0812 21:04:28.985641 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:04:28.985642 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:04:28.985651 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:04:28.985705 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:04:28.985866 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:28.986574 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:28.987806 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:28.988363 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:28.988441 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:28.988449 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 21:04:28.988756 (+ 307us) server_negotiation.cc:299] Negotiation successful
0812 21:04:28.988803 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260812 21:04:28.989077 30853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:28.985266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.130:41989 (local address 127.11.254.131:55597)
0812 21:04:28.985468 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0812 21:04:28.985477 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 21:04:28.985583 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:04:28.985743 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:04:28.985743 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:04:28.985744 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:04:28.985813 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:28.985815 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:28.986643 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:28.986643 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:28.987731 (+ 1088us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:04:28.987733 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:04:28.988612 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:04:28.988613 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:04:28.988671 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:28.988674 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 21:04:28.988970 (+ 296us) client_negotiation.cc:240] Negotiation successful
0812 21:04:28.989008 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":129,"threads_started":1}
I20260812 21:04:28.989178 30322 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "483cc9bbe4a34f44b907b711f75b6070" candidate_uuid: "75433f0931d34bed963cb1cca3a74146" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8287c2babb474575be99f0623283c4c1" is_pre_election: true
I20260812 21:04:28.989362 30322 raft_consensus.cc:2454] T 483cc9bbe4a34f44b907b711f75b6070 P 8287c2babb474575be99f0623283c4c1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 75433f0931d34bed963cb1cca3a74146 in term 0.
I20260812 21:04:28.989390 30533 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "483cc9bbe4a34f44b907b711f75b6070" candidate_uuid: "75433f0931d34bed963cb1cca3a74146" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e19c8a9b7ad74aca89883d87023b5c0b" is_pre_election: true
I20260812 21:04:28.989562 30533 raft_consensus.cc:2454] T 483cc9bbe4a34f44b907b711f75b6070 P e19c8a9b7ad74aca89883d87023b5c0b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 75433f0931d34bed963cb1cca3a74146 in term 0.
I20260812 21:04:28.989570 30638 leader_election.cc:302] T 483cc9bbe4a34f44b907b711f75b6070 P 75433f0931d34bed963cb1cca3a74146 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 75433f0931d34bed963cb1cca3a74146, 8287c2babb474575be99f0623283c4c1; no voters:
I20260812 21:04:28.989682 30848 raft_consensus.cc:2781] T 483cc9bbe4a34f44b907b711f75b6070 P 75433f0931d34bed963cb1cca3a74146 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 21:04:28.989754 30848 raft_consensus.cc:482] T 483cc9bbe4a34f44b907b711f75b6070 P 75433f0931d34bed963cb1cca3a74146 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 21:04:28.989776 30848 raft_consensus.cc:3037] T 483cc9bbe4a34f44b907b711f75b6070 P 75433f0931d34bed963cb1cca3a74146 [term 0 FOLLOWER]: Advancing to term 1
W20260812 21:04:28.990286 30603 tablet.cc:2369] T 483cc9bbe4a34f44b907b711f75b6070 P e19c8a9b7ad74aca89883d87023b5c0b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 21:04:28.990727 30848 raft_consensus.cc:504] T 483cc9bbe4a34f44b907b711f75b6070 P 75433f0931d34bed963cb1cca3a74146 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8287c2babb474575be99f0623283c4c1" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 42693 } } peers { permanent_uuid: "75433f0931d34bed963cb1cca3a74146" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 43887 } } peers { permanent_uuid: "e19c8a9b7ad74aca89883d87023b5c0b" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 41989 } }
I20260812 21:04:28.990844 30848 leader_election.cc:288] T 483cc9bbe4a34f44b907b711f75b6070 P 75433f0931d34bed963cb1cca3a74146 [CANDIDATE]: Term 1 election: Requested vote from peers 8287c2babb474575be99f0623283c4c1 (127.11.254.129:42693), e19c8a9b7ad74aca89883d87023b5c0b (127.11.254.130:41989)
I20260812 21:04:28.991014 30533 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "483cc9bbe4a34f44b907b711f75b6070" candidate_uuid: "75433f0931d34bed963cb1cca3a74146" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e19c8a9b7ad74aca89883d87023b5c0b"
I20260812 21:04:28.991024 30322 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "483cc9bbe4a34f44b907b711f75b6070" candidate_uuid: "75433f0931d34bed963cb1cca3a74146" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8287c2babb474575be99f0623283c4c1"
I20260812 21:04:28.991086 30533 raft_consensus.cc:3037] T 483cc9bbe4a34f44b907b711f75b6070 P e19c8a9b7ad74aca89883d87023b5c0b [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:04:28.991093 30322 raft_consensus.cc:3037] T 483cc9bbe4a34f44b907b711f75b6070 P 8287c2babb474575be99f0623283c4c1 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:04:28.991796 30533 raft_consensus.cc:2454] T 483cc9bbe4a34f44b907b711f75b6070 P e19c8a9b7ad74aca89883d87023b5c0b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75433f0931d34bed963cb1cca3a74146 in term 1.
I20260812 21:04:28.991852 30322 raft_consensus.cc:2454] T 483cc9bbe4a34f44b907b711f75b6070 P 8287c2babb474575be99f0623283c4c1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75433f0931d34bed963cb1cca3a74146 in term 1.
I20260812 21:04:28.991943 30638 leader_election.cc:302] T 483cc9bbe4a34f44b907b711f75b6070 P 75433f0931d34bed963cb1cca3a74146 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 75433f0931d34bed963cb1cca3a74146, e19c8a9b7ad74aca89883d87023b5c0b; no voters:
I20260812 21:04:28.992049 30848 raft_consensus.cc:2781] T 483cc9bbe4a34f44b907b711f75b6070 P 75433f0931d34bed963cb1cca3a74146 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:04:28.992233 30848 raft_consensus.cc:686] T 483cc9bbe4a34f44b907b711f75b6070 P 75433f0931d34bed963cb1cca3a74146 [term 1 LEADER]: Becoming Leader. State: Replica: 75433f0931d34bed963cb1cca3a74146, State: Running, Role: LEADER
I20260812 21:04:28.992326 30848 consensus_queue.cc:237] T 483cc9bbe4a34f44b907b711f75b6070 P 75433f0931d34bed963cb1cca3a74146 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8287c2babb474575be99f0623283c4c1" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 42693 } } peers { permanent_uuid: "75433f0931d34bed963cb1cca3a74146" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 43887 } } peers { permanent_uuid: "e19c8a9b7ad74aca89883d87023b5c0b" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 41989 } }
I20260812 21:04:28.993216 30075 catalog_manager.cc:5495] T 483cc9bbe4a34f44b907b711f75b6070 P 75433f0931d34bed963cb1cca3a74146 reported cstate change: term changed from 0 to 1, leader changed from <none> to 75433f0931d34bed963cb1cca3a74146 (127.11.254.131). New cstate: current_term: 1 leader_uuid: "75433f0931d34bed963cb1cca3a74146" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8287c2babb474575be99f0623283c4c1" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 42693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "75433f0931d34bed963cb1cca3a74146" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 43887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e19c8a9b7ad74aca89883d87023b5c0b" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 41989 } health_report { overall_health: UNKNOWN } } }
I20260812 21:04:29.008453 30838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:29.006059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40578 (local address 127.11.254.131:43887)
0812 21:04:29.006123 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0812 21:04:29.006125 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:04:29.006134 (+ 9us) server_negotiation.cc:407] Connection header received
0812 21:04:29.006202 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:04:29.006203 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:04:29.006225 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:04:29.006292 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 21:04:29.006435 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:29.006931 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:29.007672 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:04:29.007878 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:04:29.007935 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:04:29.008098 (+ 163us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 21:04:29.008213 (+ 115us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 21:04:29.008304 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0812 21:04:29.008322 (+ 18us) server_negotiation.cc:299] Negotiation successful
0812 21:04:29.008347 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260812 21:04:29.010522 30322 raft_consensus.cc:1261] T 483cc9bbe4a34f44b907b711f75b6070 P 8287c2babb474575be99f0623283c4c1 [term 1 FOLLOWER]: Refusing update from remote peer 75433f0931d34bed963cb1cca3a74146: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 21:04:29.010522 30533 raft_consensus.cc:1261] T 483cc9bbe4a34f44b907b711f75b6070 P e19c8a9b7ad74aca89883d87023b5c0b [term 1 FOLLOWER]: Refusing update from remote peer 75433f0931d34bed963cb1cca3a74146: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 21:04:29.010932 30855 consensus_queue.cc:1035] T 483cc9bbe4a34f44b907b711f75b6070 P 75433f0931d34bed963cb1cca3a74146 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e19c8a9b7ad74aca89883d87023b5c0b" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 41989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 21:04:29.011121 30848 consensus_queue.cc:1035] T 483cc9bbe4a34f44b907b711f75b6070 P 75433f0931d34bed963cb1cca3a74146 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8287c2babb474575be99f0623283c4c1" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 42693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 21:04:29.014037 30866 mvcc.cc:205] Tried to move back new op lower bound from 7317785268263833600 to 7317785268193210368. Current Snapshot: MvccSnapshot[applied={T|T < 7317785268263833600}]
I20260812 21:04:29.014322 30864 mvcc.cc:205] Tried to move back new op lower bound from 7317785268263833600 to 7317785268193210368. Current Snapshot: MvccSnapshot[applied={T|T < 7317785268263833600}]
I20260812 21:04:29.016424 30075 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59018:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 21:04:29.016525 30075 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59018:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 21:04:29.017318 30075 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1f98b9482d3b48da9f25fea1e3019bf3: Sending DeleteTablet for 3 replicas of tablet 483cc9bbe4a34f44b907b711f75b6070
I20260812 21:04:29.017741 30715 tablet_service.cc:1514] Processing DeleteTablet for tablet 483cc9bbe4a34f44b907b711f75b6070 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:04:29 UTC) from {username='kudu', principal='kudu/127.11.254.190@KRBTEST.COM'} at 127.0.0.1:40576
I20260812 21:04:29.017741 30496 tablet_service.cc:1514] Processing DeleteTablet for tablet 483cc9bbe4a34f44b907b711f75b6070 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:04:29 UTC) from {username='kudu', principal='kudu/127.11.254.190@KRBTEST.COM'} at 127.0.0.1:45478
I20260812 21:04:29.018172 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 30180
I20260812 21:04:29.018220 30286 tablet_service.cc:1514] Processing DeleteTablet for tablet 483cc9bbe4a34f44b907b711f75b6070 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:04:29 UTC) from {username='kudu', principal='kudu/127.11.254.190@KRBTEST.COM'} at 127.0.0.1:43284
I20260812 21:04:29.026240 30881 tablet_replica.cc:330] T 483cc9bbe4a34f44b907b711f75b6070 P 75433f0931d34bed963cb1cca3a74146: stopping tablet replica
I20260812 21:04:29.026422 30881 raft_consensus.cc:2229] T 483cc9bbe4a34f44b907b711f75b6070 P 75433f0931d34bed963cb1cca3a74146 [term 1 LEADER]: Raft consensus shutting down.
I20260812 21:04:29.026551 30881 raft_consensus.cc:2258] T 483cc9bbe4a34f44b907b711f75b6070 P 75433f0931d34bed963cb1cca3a74146 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 21:04:29.026742 30883 tablet_replica.cc:330] T 483cc9bbe4a34f44b907b711f75b6070 P e19c8a9b7ad74aca89883d87023b5c0b: stopping tablet replica
I20260812 21:04:29.026862 30883 raft_consensus.cc:2229] T 483cc9bbe4a34f44b907b711f75b6070 P e19c8a9b7ad74aca89883d87023b5c0b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 21:04:29.026970 30883 raft_consensus.cc:2258] T 483cc9bbe4a34f44b907b711f75b6070 P e19c8a9b7ad74aca89883d87023b5c0b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 21:04:29.027391 30881 ts_tablet_manager.cc:1905] T 483cc9bbe4a34f44b907b711f75b6070 P 75433f0931d34bed963cb1cca3a74146: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 21:04:29.027428 30883 ts_tablet_manager.cc:1905] T 483cc9bbe4a34f44b907b711f75b6070 P e19c8a9b7ad74aca89883d87023b5c0b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 21:04:29.029265 30883 ts_tablet_manager.cc:1918] T 483cc9bbe4a34f44b907b711f75b6070 P e19c8a9b7ad74aca89883d87023b5c0b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 21:04:29.029349 30883 log.cc:1196] T 483cc9bbe4a34f44b907b711f75b6070 P e19c8a9b7ad74aca89883d87023b5c0b: Deleting WAL directory at /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568607354921-12282-0/minicluster-data/ts-1/wal/wals/483cc9bbe4a34f44b907b711f75b6070
I20260812 21:04:29.029278 30881 ts_tablet_manager.cc:1918] T 483cc9bbe4a34f44b907b711f75b6070 P 75433f0931d34bed963cb1cca3a74146: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 21:04:29.029528 30881 log.cc:1196] T 483cc9bbe4a34f44b907b711f75b6070 P 75433f0931d34bed963cb1cca3a74146: Deleting WAL directory at /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786568607354921-12282-0/minicluster-data/ts-2/wal/wals/483cc9bbe4a34f44b907b711f75b6070
I20260812 21:04:29.029608 30883 ts_tablet_manager.cc:1939] T 483cc9bbe4a34f44b907b711f75b6070 P e19c8a9b7ad74aca89883d87023b5c0b: Deleting consensus metadata
I20260812 21:04:29.029896 30881 ts_tablet_manager.cc:1939] T 483cc9bbe4a34f44b907b711f75b6070 P 75433f0931d34bed963cb1cca3a74146: Deleting consensus metadata
I20260812 21:04:29.030148 30046 catalog_manager.cc:4841] TS e19c8a9b7ad74aca89883d87023b5c0b (127.11.254.130:41989): tablet 483cc9bbe4a34f44b907b711f75b6070 (table test-table [id=2dc0cadf1faf429aab1ed0b72984e35b]) successfully deleted
W20260812 21:04:29.030546 30046 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.254.129:42693 (error 108)
I20260812 21:04:29.030758 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 30397
I20260812 21:04:29.030844 30046 catalog_manager.cc:4841] TS 75433f0931d34bed963cb1cca3a74146 (127.11.254.131:43887): tablet 483cc9bbe4a34f44b907b711f75b6070 (table test-table [id=2dc0cadf1faf429aab1ed0b72984e35b]) successfully deleted
I20260812 21:04:29.034164 30834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:04:29.031003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.129:42693 (local address 127.0.0.1:43292)
0812 21:04:29.034061 (+ 3058us) negotiation.cc:107] Waiting for socket to connect
0812 21:04:29.034126 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.129:42693: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2971}
W20260812 21:04:29.036939 30046 catalog_manager.cc:4584] TS 8287c2babb474575be99f0623283c4c1 (127.11.254.129:42693): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 483cc9bbe4a34f44b907b711f75b6070: Network error: Client connection negotiation failed: client connection to 127.11.254.129:42693: connect: Connection refused (error 111)
I20260812 21:04:29.042295 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 30609
I20260812 21:04:29.061853 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 30023
2026-08-12T21:04:29Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2101 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568607354921-12282-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 21:04:29 dist-test-slave-vpvm krb5kdc[30887](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 21:04:29 dist-test-slave-vpvm krb5kdc[30887](info): set up 2 sockets
Aug 12 21:04:29 dist-test-slave-vpvm krb5kdc[30887](info): commencing operation
krb5kdc: starting...
W20260812 21:04:30.252509 12282 mini_kdc.cc:121] Time spent starting KDC: real 1.163s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568607354921-12282-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568607354921-12282-0/krb5kdc/test-user.keytab.
Aug 12 21:04:30 dist-test-slave-vpvm krb5kdc[30887](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568670, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T21:04:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T21:04:30Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.11.254.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.11.254.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.11.254.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568607354921-12282-0/krb5kdc/rangeradmin_127.11.254.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.11.254.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568607354921-12282-0/krb5kdc/rangeradmin_127.11.254.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.11.254.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.11.254.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.11.254.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568607354921-12282-0/krb5kdc/rangerlookup_127.11.254.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.11.254.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568607354921-12282-0/krb5kdc/rangerlookup_127.11.254.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.11.254.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.11.254.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.11.254.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568607354921-12282-0/krb5kdc/HTTP_127.11.254.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.11.254.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568607354921-12282-0/krb5kdc/HTTP_127.11.254.148@KRBTEST.COM.keytab.
I20260812 21:04:30.428968 12282 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568607354921-12282-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskymWbKN/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568607354921-12282-0/minicluster-data/postgres -l logfile start
2026-08-12 21:04:33.002 UTC [30933] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-12 21:04:33.002 UTC [30933] LOG: listening on IPv4 address "127.11.254.148", port 36497
2026-08-12 21:04:33.005 UTC [30933] LOG: listening on Unix socket "/tmp/.s.PGSQL.36497"
2026-08-12 21:04:33.027 UTC [30939] LOG: database system was shut down at 2026-08-12 21:04:31 UTC
2026-08-12 21:04:33.030 UTC [30933] LOG: database system is ready to accept connections
I20260812 21:04:34.135790 12282 mini_postgres.cc:96] Postgres bound to 36497
2026-08-12 21:04:34.145 UTC [30958] FATAL: database "slave" does not exist
127.11.254.148:36497 - accepting connections
I20260812 21:04:34.145937 12282 mini_ranger.cc:161] Starting Ranger...
I20260812 21:04:34.163268 12282 mini_ranger.cc:84] Created miniranger Postgres user
I20260812 21:04:34.637866 12282 mini_ranger.cc:87] Created ranger Postgres database
I20260812 21:04:34.637979 12282 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568607354921-12282-0/minicluster-data/ranger-admin
2026-08-12 21:04:35,002 [I] DB FLAVOR :POSTGRES
2026-08-12 21:04:35,002 [I] --------- Verifying Ranger DB connection ---------
2026-08-12 21:04:35,002 [I] Checking connection..
2026-08-12 21:04:35,003 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskymWbKN/build/release/bin/postgresql.jar:/tmp/dist-test-taskymWbKN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.254.148:36497/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-12 21:04:35,280 [I] Checking connection passed.
2026-08-12 21:04:35,280 [I] --------- Verifying version history table ---------
2026-08-12 21:04:35,280 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskymWbKN/build/release/bin/postgresql.jar:/tmp/dist-test-taskymWbKN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.254.148:36497/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 21:04:35,549 [I] Table x_db_version_h does not exist in database ranger
2026-08-12 21:04:35,549 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskymWbKN/build/release/bin/postgresql.jar:/tmp/dist-test-taskymWbKN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.254.148:36497/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 21:04:35,807 [I] Table x_db_version_h does not exist in database ranger
2026-08-12 21:04:35,808 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-12 21:04:35,808 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskymWbKN/build/release/bin/postgresql.jar:/tmp/dist-test-taskymWbKN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.254.148:36497/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskymWbKN/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-12 21:04:36.071 UTC [31053] WARNING: there is no transaction in progress
2026-08-12 21:04:36,078 [I] create_dbversion_catalog.sql file imported successfully
2026-08-12 21:04:36,078 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskymWbKN/build/release/bin/postgresql.jar:/tmp/dist-test-taskymWbKN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.254.148:36497/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 21:04:36,339 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-12 21:04:36,339 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-12 21:04:36,340 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskymWbKN/build/release/bin/postgresql.jar:/tmp/dist-test-taskymWbKN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.254.148:36497/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-12 21:04:36,597 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskymWbKN/build/release/bin/postgresql.jar:/tmp/dist-test-taskymWbKN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.254.148:36497/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-12 21:04:36,842 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskymWbKN/build/release/bin/postgresql.jar:/tmp/dist-test-taskymWbKN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.254.148:36497/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-vpvm.c.gcp-upstream.internal','N') ;"
2026-08-12 21:04:37,105 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskymWbKN/build/release/bin/postgresql.jar:/tmp/dist-test-taskymWbKN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.254.148:36497/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-12 21:04:37,374 [I] Table x_portal_user does not exist in database ranger
2026-08-12 21:04:37,374 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskymWbKN/build/release/bin/postgresql.jar:/tmp/dist-test-taskymWbKN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.254.148:36497/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-12 21:04:37,648 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-12 21:04:37,649 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskymWbKN/build/release/bin/postgresql.jar:/tmp/dist-test-taskymWbKN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.254.148:36497/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-12 21:04:37,922 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskymWbKN/build/release/bin/postgresql.jar:/tmp/dist-test-taskymWbKN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.254.148:36497/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-12 21:04:38,197 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-12 21:04:38,197 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskymWbKN/build/release/bin/postgresql.jar:/tmp/dist-test-taskymWbKN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.254.148:36497/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskymWbKN/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-12 21:04:39.193 UTC [31232] WARNING: there is no transaction in progress
2026-08-12 21:04:39.212 UTC [31232] WARNING: there is no transaction in progress
2026-08-12 21:04:39.229 UTC [31232] WARNING: there is no transaction in progress
2026-08-12 21:04:39.246 UTC [31232] WARNING: there is no transaction in progress
2026-08-12 21:04:39.266 UTC [31232] WARNING: there is no transaction in progress
2026-08-12 21:04:39.288 UTC [31232] WARNING: there is no transaction in progress
2026-08-12 21:04:39.358 UTC [31232] WARNING: there is no transaction in progress
2026-08-12 21:04:39.366 UTC [31232] WARNING: there is no transaction in progress
2026-08-12 21:04:39.382 UTC [31232] WARNING: there is no transaction in progress
2026-08-12 21:04:39.394 UTC [31232] WARNING: there is no transaction in progress
2026-08-12 21:04:39.406 UTC [31232] WARNING: there is no transaction in progress
2026-08-12 21:04:39.420 UTC [31232] WARNING: there is no transaction in progress
2026-08-12 21:04:39.430 UTC [31232] WARNING: there is no transaction in progress
2026-08-12 21:04:39.437 UTC [31232] WARNING: there is no transaction in progress
2026-08-12 21:04:39.444 UTC [31232] WARNING: there is no transaction in progress
2026-08-12 21:04:39.451 UTC [31232] WARNING: there is no transaction in progress
2026-08-12 21:04:40,149 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-12 21:04:40,149 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskymWbKN/build/release/bin/postgresql.jar:/tmp/dist-test-taskymWbKN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.254.148:36497/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-12 21:04:40,404 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-12 21:04:40,404 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskymWbKN/build/release/bin/postgresql.jar:/tmp/dist-test-taskymWbKN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.254.148:36497/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-12 21:04:40,652 [I] Table x_portal_user already exists in database 'ranger'
2026-08-12 21:04:40,653 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskymWbKN/build/release/bin/postgresql.jar:/tmp/dist-test-taskymWbKN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.254.148:36497/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-12 21:04:40,912 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-12 21:04:40,913 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskymWbKN/build/release/bin/postgresql.jar:/tmp/dist-test-taskymWbKN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.254.148:36497/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-12 21:04:41,164 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskymWbKN/build/release/bin/postgresql.jar:/tmp/dist-test-taskymWbKN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.254.148:36497/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-12 21:04:41,430 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskymWbKN/build/release/bin/postgresql.jar:/tmp/dist-test-taskymWbKN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.254.148:36497/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-vpvm.c.gcp-upstream.internal';"
2026-08-12 21:04:41,680 [I] CORE_DB_SCHEMA import status has been updated
2026-08-12 21:04:41,681 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskymWbKN/build/release/bin/postgresql.jar:/tmp/dist-test-taskymWbKN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.254.148:36497/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-12 21:04:41,939 [I] DB_PATCHES have already been applied
I20260812 21:04:41.945322 12282 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568607354921-12282-0/minicluster-data/ranger-admin:/tmp/dist-test-taskymWbKN/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskymWbKN/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskymWbKN/build/release/bin/postgresql.jar:/tmp/dist-test-taskymWbKN/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260812 21:04:41.945401 12282 mini_ranger.cc:193] Using host: 127.11.254.148
I20260812 21:04:41.949501 12282 mini_ranger.cc:227] Ranger admin URL: http://127.11.254.148:44121
Aug 12, 2026 9:04:42 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 12, 2026 9:04:42 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 12, 2026 9:04:42 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568607354921-12282-0/minicluster-data/ranger-admin/ews/webapp
Aug 12, 2026 9:04:42 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568607354921-12282-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 12, 2026 9:04:42 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568607354921-12282-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 12, 2026 9:04:43 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568607354921-12282-0/minicluster-data/ranger-admin/ews/webapp].
Aug 12, 2026 9:04:43 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260812 21:04:43.234714 12282 mini_ranger.cc:160] Time spent starting Ranger: real 9.089s user 0.003s sys 0.009s
Aug 12, 2026 9:05:05 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 12, 2026 9:05:05 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 12, 2026 9:05:05 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 12, 2026 9:05:05 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 12, 2026 9:05:05 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 12, 2026 9:05:05 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 12, 2026 9:05:05 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 12, 2026 9:05:05 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 12, 2026 9:05:05 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 12, 2026 9:05:05 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 12, 2026 9:05:05 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 12, 2026 9:05:05 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 12, 2026 9:05:05 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 12, 2026 9:05:05 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 12, 2026 9:05:05 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 12, 2026 9:05:05 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 12, 2026 9:05:05 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 12, 2026 9:05:06 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260812 21:05:07.034929 12282 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.11.254.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.11.254.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.11.254.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568607354921-12282-0/krb5kdc/rangerkms_127.11.254.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.11.254.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568607354921-12282-0/krb5kdc/rangerkms_127.11.254.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.11.254.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.11.254.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.11.254.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568607354921-12282-0/krb5kdc/HTTP_127.11.254.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.11.254.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568607354921-12282-0/krb5kdc/HTTP_127.11.254.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 12 21:05:07 dist-test-slave-vpvm krb5kdc[30887](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568707, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260812 21:05:07.135349 12282 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260812 21:05:07.146606 12282 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260812 21:05:08.063993 12282 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260812 21:05:08.064078 12282 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568607354921-12282-0/minicluster-data/ranger-kms
I20260812 21:05:08.064096 12282 mini_ranger_kms.cc:227] Using postgres at 127.11.254.148:36497
2026-08-12 21:05:08,835 [I] DB FLAVOR :POSTGRES
2026-08-12 21:05:08,835 [I] --------- Verifying Ranger DB connection ---------
2026-08-12 21:05:08,835 [I] Checking connection
2026-08-12 21:05:08,835 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskymWbKN/build/release/bin/postgresql.jar:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568607354921-12282-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.254.148:36497/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-12 21:05:09,214 [I] connection success
2026-08-12 21:05:09,215 [I] --------- Verifying Ranger DB tables ---------
2026-08-12 21:05:09,215 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-12 21:05:09,215 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskymWbKN/build/release/bin/postgresql.jar:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568607354921-12282-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.254.148:36497/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-12 21:05:09,462 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-12 21:05:09,462 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-12 21:05:09,462 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-12 21:05:09,462 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskymWbKN/build/release/bin/postgresql.jar:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568607354921-12282-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.254.148:36497/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568607354921-12282-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-12 21:05:09,756 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260812 21:05:10.233867 12282 mini_ranger_kms.cc:310] Created kms service
I20260812 21:05:10.477414 12282 mini_ranger_kms.cc:326] Created kudu user
I20260812 21:05:10.524814 12282 mini_ranger_kms.cc:343] Created rangerkms user
Aug 12 21:05:10 dist-test-slave-vpvm krb5kdc[30887](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568707, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.11.254.148@KRBTEST.COM
I20260812 21:05:10.955279 12282 mini_ranger_kms.cc:384] Added ranger policy
I20260812 21:05:10.955615 12282 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568607354921-12282-0/minicluster-data/ranger-kms:/tmp/dist-test-taskymWbKN/build/release/bin/postgresql.jar:/tmp/dist-test-taskymWbKN/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskymWbKN/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260812 21:05:10.955644 12282 mini_ranger_kms.cc:240] Using host: 127.11.254.148
I20260812 21:05:10.960397 12282 mini_ranger_kms.cc:276] Ranger KMS PID: 31542
I20260812 21:05:10.960503 12282 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.11.254.148:54335
Aug 12, 2026 9:05:12 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 12, 2026 9:05:12 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 12, 2026 9:05:12 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568607354921-12282-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 12, 2026 9:05:12 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568607354921-12282-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 12, 2026 9:05:12 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568607354921-12282-0/minicluster-data/ranger-kms/ews/webapp].
Aug 12, 2026 9:05:12 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260812 21:05:13.496969 12282 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.362s user 0.006s sys 0.728s
I20260812 21:05:13.497087 12282 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260812 21:05:13.497164 12282 mini_ranger_kms.cc:401] 127.11.254.148:54335/kms/v1/keys
Aug 12 21:05:13 dist-test-slave-vpvm krb5kdc[30887](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786568713, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.11.254.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 12 21:05:14 dist-test-slave-vpvm krb5kdc[30887](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568713, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.11.254.148@KRBTEST.COM for HTTP/127.11.254.148@KRBTEST.COM
Aug 12 21:05:14 dist-test-slave-vpvm krb5kdc[30887](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568713, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.11.254.148@KRBTEST.COM for HTTP/127.11.254.148@KRBTEST.COM
Aug 12 21:05:17 dist-test-slave-vpvm krb5kdc[30887](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568717, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.11.254.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.254.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.254.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.190.keytab.
Entry for principal kudu/127.11.254.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.190.keytab.
I20260812 21:05:17.357411 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568607354921-12282-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568607354921-12282-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568607354921-12282-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.254.190:45247
--webserver_interface=127.11.254.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.190.keytab
--principal=kudu/127.11.254.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.254.148:35097
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.254.190:45247
--ranger_config_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568607354921-12282-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.254.148:54335/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568607354921-12282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568607354921-12282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568607354921-12282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:05:17.460497 31629 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:05:17.460733 31629 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:05:17.460767 31629 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:05:17.462085 31629 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 21:05:17.462131 31629 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:05:17.462150 31629 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:05:17.462165 31629 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 21:05:17.462193 31629 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 21:05:17.464274 31629 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.254.148:35097
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568607354921-12282-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568607354921-12282-0/minicluster-data/master-0/wal
--ranger_kms_url=127.11.254.148:54335/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.11.254.190:45247
--ranger_config_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568607354921-12282-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.190.keytab
--principal=kudu/127.11.254.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.254.190:45247
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.254.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31629
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568607354921-12282-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:05:17.464562 31629 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:05:17.464802 31629 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:05:17.470762 31640 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:05:17.470718 31635 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:05:17.470943 31629 server_base.cc:1034] running on GCE node
W20260812 21:05:17.470854 31637 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:05:17.471329 31629 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:05:17.471643 31629 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:05:17.472818 31629 hybrid_clock.cc:648] HybridClock initialized: now 1786568717472794 us; error 37 us; skew 500 ppm
Aug 12 21:05:17 dist-test-slave-vpvm krb5kdc[30887](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568717, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:05:17.475514 31629 init.cc:377] Logged in from keytab as kudu/127.11.254.190@KRBTEST.COM (short username kudu)
I20260812 21:05:17.475960 31629 webserver.cc:460] Webserver started at http://127.11.254.190:35083/ using document root <none> and password file <none>
I20260812 21:05:17.476183 31629 fs_manager.cc:359] Metadata directory not provided
I20260812 21:05:17.476225 31629 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:05:17.476357 31629 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 21:05:17 dist-test-slave-vpvm krb5kdc[30887](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568717, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.190@KRBTEST.COM for HTTP/127.11.254.148@KRBTEST.COM
I20260812 21:05:17.534411 31629 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568607354921-12282-0/minicluster-data/master-0/data/instance:
uuid: "9c0b48d1b6fd40cebe4b26cd9d04d0e2"
format_stamp: "Formatted at 2026-08-12 21:05:17 on dist-test-slave-vpvm"
server_key: "e07f801f089b4b575a2ccf1d6d057834"
server_key_iv: "19e6db3b7319e5c5cfe5d8d9533c970e"
server_key_version: "kuduclusterkey@0"
I20260812 21:05:17.535018 31629 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568607354921-12282-0/minicluster-data/master-0/wal/instance:
uuid: "9c0b48d1b6fd40cebe4b26cd9d04d0e2"
format_stamp: "Formatted at 2026-08-12 21:05:17 on dist-test-slave-vpvm"
server_key: "e07f801f089b4b575a2ccf1d6d057834"
server_key_iv: "19e6db3b7319e5c5cfe5d8d9533c970e"
server_key_version: "kuduclusterkey@0"
I20260812 21:05:17.537103 31629 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.001s
I20260812 21:05:17.552630 31652 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:05:17.552884 31629 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 21:05:17.553014 31629 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568607354921-12282-0/minicluster-data/master-0/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568607354921-12282-0/minicluster-data/master-0/wal
uuid: "9c0b48d1b6fd40cebe4b26cd9d04d0e2"
format_stamp: "Formatted at 2026-08-12 21:05:17 on dist-test-slave-vpvm"
server_key: "e07f801f089b4b575a2ccf1d6d057834"
server_key_iv: "19e6db3b7319e5c5cfe5d8d9533c970e"
server_key_version: "kuduclusterkey@0"
I20260812 21:05:17.553117 31629 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568607354921-12282-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568607354921-12282-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568607354921-12282-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:05:17.561944 31629 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:05:17.564260 31629 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:05:17.564543 31629 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:05:17.575271 31629 rpc_server.cc:307] RPC server started. Bound to: 127.11.254.190:45247
I20260812 21:05:17.575371 31727 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.254.190:45247 every 8 connection(s)
I20260812 21:05:17.575786 31629 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
I20260812 21:05:17.576694 31731 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:05:17.580142 31731 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9c0b48d1b6fd40cebe4b26cd9d04d0e2: Bootstrap starting.
I20260812 21:05:17.581307 31731 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9c0b48d1b6fd40cebe4b26cd9d04d0e2: Neither blocks nor log segments found. Creating new log.
I20260812 21:05:17.582175 31731 log.cc:824] T 00000000000000000000000000000000 P 9c0b48d1b6fd40cebe4b26cd9d04d0e2: Log is configured to *not* fsync() on all Append() calls
I20260812 21:05:17.582955 31731 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9c0b48d1b6fd40cebe4b26cd9d04d0e2: No bootstrap required, opened a new log
I20260812 21:05:17.584885 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 31629
I20260812 21:05:17.584964 12282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568607354921-12282-0/minicluster-data/master-0/wal/instance
I20260812 21:05:17.585150 12282 external_mini_cluster.cc:1428] Setting key ca55aa3522b1617d7006e537472f521e
I20260812 21:05:17.586920 31731 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9c0b48d1b6fd40cebe4b26cd9d04d0e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c0b48d1b6fd40cebe4b26cd9d04d0e2" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 45247 } }
I20260812 21:05:17.587073 31731 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9c0b48d1b6fd40cebe4b26cd9d04d0e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:05:17.587106 31731 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9c0b48d1b6fd40cebe4b26cd9d04d0e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c0b48d1b6fd40cebe4b26cd9d04d0e2, State: Initialized, Role: FOLLOWER
I20260812 21:05:17.587245 31731 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9c0b48d1b6fd40cebe4b26cd9d04d0e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c0b48d1b6fd40cebe4b26cd9d04d0e2" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 45247 } }
I20260812 21:05:17.587339 31731 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9c0b48d1b6fd40cebe4b26cd9d04d0e2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 21:05:17.587373 31731 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9c0b48d1b6fd40cebe4b26cd9d04d0e2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 21:05:17.587425 31731 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9c0b48d1b6fd40cebe4b26cd9d04d0e2 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:05:17.588507 31731 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9c0b48d1b6fd40cebe4b26cd9d04d0e2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c0b48d1b6fd40cebe4b26cd9d04d0e2" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 45247 } }
I20260812 21:05:17.588609 31731 leader_election.cc:302] T 00000000000000000000000000000000 P 9c0b48d1b6fd40cebe4b26cd9d04d0e2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9c0b48d1b6fd40cebe4b26cd9d04d0e2; no voters:
I20260812 21:05:17.588773 31731 leader_election.cc:288] T 00000000000000000000000000000000 P 9c0b48d1b6fd40cebe4b26cd9d04d0e2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 21:05:17.588824 31743 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9c0b48d1b6fd40cebe4b26cd9d04d0e2 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:05:17.589025 31731 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9c0b48d1b6fd40cebe4b26cd9d04d0e2 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 21:05:17.589052 31743 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9c0b48d1b6fd40cebe4b26cd9d04d0e2 [term 1 LEADER]: Becoming Leader. State: Replica: 9c0b48d1b6fd40cebe4b26cd9d04d0e2, State: Running, Role: LEADER
I20260812 21:05:17.589229 31743 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9c0b48d1b6fd40cebe4b26cd9d04d0e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c0b48d1b6fd40cebe4b26cd9d04d0e2" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 45247 } }
I20260812 21:05:17.589745 31744 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c0b48d1b6fd40cebe4b26cd9d04d0e2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9c0b48d1b6fd40cebe4b26cd9d04d0e2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c0b48d1b6fd40cebe4b26cd9d04d0e2" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 45247 } } }
I20260812 21:05:17.589852 31744 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c0b48d1b6fd40cebe4b26cd9d04d0e2 [sys.catalog]: This master's current role is: LEADER
I20260812 21:05:17.589913 31745 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c0b48d1b6fd40cebe4b26cd9d04d0e2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9c0b48d1b6fd40cebe4b26cd9d04d0e2. Latest consensus state: current_term: 1 leader_uuid: "9c0b48d1b6fd40cebe4b26cd9d04d0e2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c0b48d1b6fd40cebe4b26cd9d04d0e2" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 45247 } } }
I20260812 21:05:17.590013 31745 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c0b48d1b6fd40cebe4b26cd9d04d0e2 [sys.catalog]: This master's current role is: LEADER
I20260812 21:05:17.590476 31751 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 21:05:17.592993 31751 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 21:05:17.594100 31731 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568607354921-12282-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
Aug 12 21:05:17 dist-test-slave-vpvm krb5kdc[30887](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568717, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.254.190@KRBTEST.COM
I20260812 21:05:17.601518 31751 catalog_manager.cc:1347] Generated new cluster ID: f5bb7e50fdea43a9b3f8a6dd0c6d1ca4
I20260812 21:05:17.601749 31751 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 21:05:17.605873 31742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:05:17.585670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34268 (local address 127.11.254.190:45247)
0812 21:05:17.585923 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0812 21:05:17.585934 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 21:05:17.586025 (+ 91us) server_negotiation.cc:407] Connection header received
0812 21:05:17.586992 (+ 967us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:05:17.587001 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:05:17.587056 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:05:17.587198 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:05:17.588109 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:05:17.590441 (+ 2332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:05:17.591401 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:05:17.592100 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:05:17.592206 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:05:17.595228 (+ 3022us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:05:17.595233 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:05:17.595233 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:05:17.595246 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:05:17.603028 (+ 7782us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:05:17.603610 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:05:17.603615 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:05:17.603615 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:05:17.603834 (+ 219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:05:17.604068 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:05:17.604070 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:05:17.604070 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:05:17.604632 (+ 562us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:05:17.605035 (+ 403us) server_negotiation.cc:1035] Waiting for connection context
0812 21:05:17.605127 (+ 92us) server_negotiation.cc:299] Negotiation successful
0812 21:05:17.605378 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":68,"threads_started":1}
I20260812 21:05:17.616780 31751 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 21:05:17.617405 31751 catalog_manager.cc:1504] Loading token signing keys...
I20260812 21:05:17.623165 31751 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9c0b48d1b6fd40cebe4b26cd9d04d0e2: Generated new TSK 0
I20260812 21:05:17.623390 31751 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.254.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.254.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.254.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.129.keytab.
Entry for principal kudu/127.11.254.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.129.keytab.
I20260812 21:05:18.501616 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568607354921-12282-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568607354921-12282-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568607354921-12282-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568607354921-12282-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.254.148:54335/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.254.129:0
--local_ip_for_outbound_sockets=127.11.254.129
--webserver_interface=127.11.254.129
--webserver_port=0
--tserver_master_addrs=127.11.254.190:45247
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.129.keytab
--principal=kudu/127.11.254.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.254.148:35097
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568607354921-12282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568607354921-12282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568607354921-12282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:05:18.655228 31812 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:05:18.655573 31812 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:05:18.655619 31812 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:05:18.657742 31812 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:05:18.657822 31812 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:05:18.657871 31812 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.254.129
I20260812 21:05:18.660689 31812 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.254.148:35097
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568607354921-12282-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568607354921-12282-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.11.254.148:54335/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.129.keytab
--principal=kudu/127.11.254.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.254.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568607354921-12282-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.254.129
--webserver_port=0
--tserver_master_addrs=127.11.254.190:45247
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31812
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.254.129
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568607354921-12282-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:05:18.661021 31812 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:05:18.661370 31812 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:05:18.669708 31827 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:05:18.669838 31825 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:05:18.670073 31821 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:05:18.670433 31812 server_base.cc:1034] running on GCE node
I20260812 21:05:18.670655 31812 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:05:18.670907 31812 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:05:18.672051 31812 hybrid_clock.cc:648] HybridClock initialized: now 1786568718672035 us; error 36 us; skew 500 ppm
Aug 12 21:05:18 dist-test-slave-vpvm krb5kdc[30887](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568718, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:05:18.675416 31812 init.cc:377] Logged in from keytab as kudu/127.11.254.129@KRBTEST.COM (short username kudu)
I20260812 21:05:18.675889 31812 webserver.cc:460] Webserver started at http://127.11.254.129:34901/ using document root <none> and password file <none>
I20260812 21:05:18.676126 31812 fs_manager.cc:359] Metadata directory not provided
I20260812 21:05:18.676172 31812 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:05:18.676285 31812 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 21:05:18 dist-test-slave-vpvm krb5kdc[30887](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568718, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.129@KRBTEST.COM for HTTP/127.11.254.148@KRBTEST.COM
I20260812 21:05:18.692711 31812 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568607354921-12282-0/minicluster-data/ts-0/data/instance:
uuid: "03bdccc09bc44ab18b1d58268e1c3936"
format_stamp: "Formatted at 2026-08-12 21:05:18 on dist-test-slave-vpvm"
server_key: "e8496c3d296b2ef63a2a9a3573c098ee"
server_key_iv: "b3901d89c1b9fce2696896b0879e2b31"
server_key_version: "kuduclusterkey@0"
I20260812 21:05:18.693264 31812 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568607354921-12282-0/minicluster-data/ts-0/wal/instance:
uuid: "03bdccc09bc44ab18b1d58268e1c3936"
format_stamp: "Formatted at 2026-08-12 21:05:18 on dist-test-slave-vpvm"
server_key: "e8496c3d296b2ef63a2a9a3573c098ee"
server_key_iv: "b3901d89c1b9fce2696896b0879e2b31"
server_key_version: "kuduclusterkey@0"
I20260812 21:05:18.695595 31812 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.004s
I20260812 21:05:18.711999 31836 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:05:18.712298 31812 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 21:05:18.712421 31812 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568607354921-12282-0/minicluster-data/ts-0/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568607354921-12282-0/minicluster-data/ts-0/wal
uuid: "03bdccc09bc44ab18b1d58268e1c3936"
format_stamp: "Formatted at 2026-08-12 21:05:18 on dist-test-slave-vpvm"
server_key: "e8496c3d296b2ef63a2a9a3573c098ee"
server_key_iv: "b3901d89c1b9fce2696896b0879e2b31"
server_key_version: "kuduclusterkey@0"
I20260812 21:05:18.712543 31812 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568607354921-12282-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568607354921-12282-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568607354921-12282-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:05:18.727373 31812 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:05:18.730170 31812 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:05:18.730439 31812 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:05:18.730899 31812 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:05:18.731384 31812 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:05:18.731441 31812 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:05:18.731472 31812 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:05:18.731496 31812 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:05:18.755633 31812 rpc_server.cc:307] RPC server started. Bound to: 127.11.254.129:37565
I20260812 21:05:18.755774 32034 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.254.129:37565 every 8 connection(s)
I20260812 21:05:18.756202 31812 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568607354921-12282-0/minicluster-data/ts-0/data/info.pb
I20260812 21:05:18.762773 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 31812
I20260812 21:05:18.762877 12282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568607354921-12282-0/minicluster-data/ts-0/wal/instance
I20260812 21:05:18.763085 12282 external_mini_cluster.cc:1428] Setting key c2634617034104dc1000b01f59eab2c4
Aug 12 21:05:18 dist-test-slave-vpvm krb5kdc[30887](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568718, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.129@KRBTEST.COM for kudu/127.11.254.190@KRBTEST.COM
I20260812 21:05:18.768389 32038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:05:18.757308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.129:36053 (local address 127.11.254.190:45247)
0812 21:05:18.757471 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0812 21:05:18.757473 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:05:18.757776 (+ 303us) server_negotiation.cc:407] Connection header received
0812 21:05:18.758659 (+ 883us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:05:18.758661 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:05:18.758677 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:05:18.758742 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:05:18.759652 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:05:18.760182 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:05:18.761318 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:05:18.761531 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:05:18.761589 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:05:18.764610 (+ 3021us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:05:18.764611 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:05:18.764612 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:05:18.764620 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:05:18.766495 (+ 1875us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:05:18.767072 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:05:18.767073 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:05:18.767073 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:05:18.767170 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:05:18.767501 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:05:18.767502 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:05:18.767502 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:05:18.767680 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:05:18.767758 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0812 21:05:18.768128 (+ 370us) server_negotiation.cc:299] Negotiation successful
0812 21:05:18.768236 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":66,"threads_started":1}
I20260812 21:05:18.768440 32039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:05:18.757286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:45247 (local address 127.11.254.129:36053)
0812 21:05:18.757624 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0812 21:05:18.757663 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 21:05:18.758555 (+ 892us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:05:18.758816 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:05:18.758820 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:05:18.759198 (+ 378us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:05:18.759554 (+ 356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:05:18.759560 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:05:18.760264 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:05:18.760265 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:05:18.761231 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:05:18.761234 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:05:18.761602 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:05:18.761603 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:05:18.761794 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:05:18.762516 (+ 722us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:05:18.762526 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:05:18.764515 (+ 1989us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:05:18.766607 (+ 2092us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:05:18.766619 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:05:18.766622 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:05:18.766973 (+ 351us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:05:18.767252 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:05:18.767253 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:05:18.767254 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:05:18.767418 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:05:18.767762 (+ 344us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:05:18.767772 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:05:18.767965 (+ 193us) client_negotiation.cc:769] Sending connection context
0812 21:05:18.768061 (+ 96us) client_negotiation.cc:240] Negotiation successful
0812 21:05:18.768195 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":209,"threads_started":1}
I20260812 21:05:18.769193 32036 heartbeater.cc:347] Connected to a master server at 127.11.254.190:45247
I20260812 21:05:18.769310 32036 heartbeater.cc:464] Registering TS with master...
I20260812 21:05:18.769569 32036 heartbeater.cc:511] Master 127.11.254.190:45247 requested a full tablet report, sending...
I20260812 21:05:18.770243 31674 ts_manager.cc:194] Registered new tserver with Master: 03bdccc09bc44ab18b1d58268e1c3936 (127.11.254.129:37565)
I20260812 21:05:18.771142 31674 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.254.129@KRBTEST.COM'} at 127.11.254.129:36053
WARNING: no policy specified for kudu/127.11.254.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.254.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.254.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.130.keytab.
Entry for principal kudu/127.11.254.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.130.keytab.
I20260812 21:05:18.801760 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568607354921-12282-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568607354921-12282-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568607354921-12282-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568607354921-12282-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.254.148:54335/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.254.130:0
--local_ip_for_outbound_sockets=127.11.254.130
--webserver_interface=127.11.254.130
--webserver_port=0
--tserver_master_addrs=127.11.254.190:45247
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.130.keytab
--principal=kudu/127.11.254.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.254.148:35097
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568607354921-12282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568607354921-12282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568607354921-12282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:05:18.981699 32048 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:05:18.982079 32048 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:05:18.982131 32048 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:05:18.984117 32048 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:05:18.984189 32048 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:05:18.984236 32048 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.254.130
I20260812 21:05:18.987186 32048 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.254.148:35097
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568607354921-12282-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568607354921-12282-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.11.254.148:54335/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.130.keytab
--principal=kudu/127.11.254.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.254.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568607354921-12282-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.254.130
--webserver_port=0
--tserver_master_addrs=127.11.254.190:45247
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32048
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.254.130
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568607354921-12282-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:05:18.987699 32048 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:05:18.988056 32048 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:05:19.001590 32059 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:05:19.001919 32060 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:05:19.002640 32048 server_base.cc:1034] running on GCE node
W20260812 21:05:19.003100 32064 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:05:19.003500 32048 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:05:19.003840 32048 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:05:19.005038 32048 hybrid_clock.cc:648] HybridClock initialized: now 1786568719005007 us; error 42 us; skew 500 ppm
Aug 12 21:05:19 dist-test-slave-vpvm krb5kdc[30887](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568719, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:05:19.009577 32048 init.cc:377] Logged in from keytab as kudu/127.11.254.130@KRBTEST.COM (short username kudu)
I20260812 21:05:19.010233 32048 webserver.cc:460] Webserver started at http://127.11.254.130:43041/ using document root <none> and password file <none>
I20260812 21:05:19.010516 32048 fs_manager.cc:359] Metadata directory not provided
I20260812 21:05:19.010586 32048 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:05:19.010792 32048 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 21:05:19 dist-test-slave-vpvm krb5kdc[30887](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568719, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.130@KRBTEST.COM for HTTP/127.11.254.148@KRBTEST.COM
I20260812 21:05:19.026515 32048 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568607354921-12282-0/minicluster-data/ts-1/data/instance:
uuid: "06aa9aeebbaf473cba89a6e94493fc93"
format_stamp: "Formatted at 2026-08-12 21:05:19 on dist-test-slave-vpvm"
server_key: "fa31b42eabbc451a83be633e9a6e75e3"
server_key_iv: "5b2b04a1832cc1f3e5c8501c4a941bd9"
server_key_version: "kuduclusterkey@0"
I20260812 21:05:19.027160 32048 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568607354921-12282-0/minicluster-data/ts-1/wal/instance:
uuid: "06aa9aeebbaf473cba89a6e94493fc93"
format_stamp: "Formatted at 2026-08-12 21:05:19 on dist-test-slave-vpvm"
server_key: "fa31b42eabbc451a83be633e9a6e75e3"
server_key_iv: "5b2b04a1832cc1f3e5c8501c4a941bd9"
server_key_version: "kuduclusterkey@0"
Aug 12 21:05:19 dist-test-slave-vpvm krb5kdc[30887](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786568719, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:05:19.040396 32048 fs_manager.cc:691] Time spent creating directory manager: real 0.013s user 0.002s sys 0.000s
I20260812 21:05:19.057796 32075 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:05:19.058068 32048 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 21:05:19.058182 32048 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568607354921-12282-0/minicluster-data/ts-1/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568607354921-12282-0/minicluster-data/ts-1/wal
uuid: "06aa9aeebbaf473cba89a6e94493fc93"
format_stamp: "Formatted at 2026-08-12 21:05:19 on dist-test-slave-vpvm"
server_key: "fa31b42eabbc451a83be633e9a6e75e3"
server_key_iv: "5b2b04a1832cc1f3e5c8501c4a941bd9"
server_key_version: "kuduclusterkey@0"
I20260812 21:05:19.058357 32048 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568607354921-12282-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568607354921-12282-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568607354921-12282-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:05:19.082222 32048 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:05:19.084926 32048 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:05:19.085186 32048 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:05:19.085655 32048 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:05:19.086186 32048 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:05:19.086248 32048 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:05:19.086282 32048 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:05:19.086306 32048 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:05:19.108125 32048 rpc_server.cc:307] RPC server started. Bound to: 127.11.254.130:35215
I20260812 21:05:19.108290 32274 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.254.130:35215 every 8 connection(s)
I20260812 21:05:19.108708 32048 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568607354921-12282-0/minicluster-data/ts-1/data/info.pb
Aug 12 21:05:19 dist-test-slave-vpvm krb5kdc[30887](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568719, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.130@KRBTEST.COM for kudu/127.11.254.190@KRBTEST.COM
I20260812 21:05:19.117969 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 32048
I20260812 21:05:19.118067 12282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568607354921-12282-0/minicluster-data/ts-1/wal/instance
I20260812 21:05:19.118273 12282 external_mini_cluster.cc:1428] Setting key d01b9e0481966f30a9944914b0445fc9
I20260812 21:05:19.121284 32038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:05:19.109935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.130:46095 (local address 127.11.254.190:45247)
0812 21:05:19.110039 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0812 21:05:19.110041 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:05:19.110496 (+ 455us) server_negotiation.cc:407] Connection header received
0812 21:05:19.111437 (+ 941us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:05:19.111439 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:05:19.111457 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:05:19.111523 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:05:19.112522 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:05:19.113047 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:05:19.114305 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:05:19.114604 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:05:19.114677 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:05:19.117446 (+ 2769us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:05:19.117448 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:05:19.117448 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:05:19.117455 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:05:19.119651 (+ 2196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:05:19.120139 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:05:19.120141 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:05:19.120141 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:05:19.120202 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:05:19.120459 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:05:19.120460 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:05:19.120460 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:05:19.120591 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:05:19.120631 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0812 21:05:19.121047 (+ 416us) server_negotiation.cc:299] Negotiation successful
0812 21:05:19.121159 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260812 21:05:19.121884 32282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:05:19.109938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:45247 (local address 127.11.254.130:46095)
0812 21:05:19.110316 (+ 378us) negotiation.cc:107] Waiting for socket to connect
0812 21:05:19.110363 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 21:05:19.111311 (+ 948us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:05:19.111601 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:05:19.111605 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:05:19.112023 (+ 418us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:05:19.112407 (+ 384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:05:19.112415 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:05:19.113144 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:05:19.113145 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:05:19.114211 (+ 1066us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:05:19.114214 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:05:19.114697 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:05:19.114698 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:05:19.114931 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:05:19.115628 (+ 697us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:05:19.115637 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:05:19.117352 (+ 1715us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:05:19.119745 (+ 2393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:05:19.119753 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:05:19.119756 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:05:19.120052 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:05:19.120268 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:05:19.120268 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:05:19.120268 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:05:19.120395 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:05:19.120674 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:05:19.120680 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:05:19.120855 (+ 175us) client_negotiation.cc:769] Sending connection context
0812 21:05:19.121390 (+ 535us) client_negotiation.cc:240] Negotiation successful
0812 21:05:19.121574 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":204,"threads_started":1}
I20260812 21:05:19.122614 32276 heartbeater.cc:347] Connected to a master server at 127.11.254.190:45247
I20260812 21:05:19.122722 32276 heartbeater.cc:464] Registering TS with master...
I20260812 21:05:19.122915 32276 heartbeater.cc:511] Master 127.11.254.190:45247 requested a full tablet report, sending...
I20260812 21:05:19.123292 31674 ts_manager.cc:194] Registered new tserver with Master: 06aa9aeebbaf473cba89a6e94493fc93 (127.11.254.130:35215)
I20260812 21:05:19.123656 31674 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.254.130@KRBTEST.COM'} at 127.11.254.130:46095
WARNING: no policy specified for kudu/127.11.254.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.254.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.254.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.131.keytab.
Entry for principal kudu/127.11.254.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.131.keytab.
I20260812 21:05:19.156276 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568607354921-12282-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568607354921-12282-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568607354921-12282-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568607354921-12282-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.254.148:54335/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.254.131:0
--local_ip_for_outbound_sockets=127.11.254.131
--webserver_interface=127.11.254.131
--webserver_port=0
--tserver_master_addrs=127.11.254.190:45247
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.131.keytab
--principal=kudu/127.11.254.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.254.148:35097
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568607354921-12282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568607354921-12282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568607354921-12282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:05:19.306015 32289 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:05:19.306245 32289 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:05:19.306283 32289 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:05:19.307703 32289 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:05:19.307750 32289 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:05:19.307789 32289 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.254.131
I20260812 21:05:19.309620 32289 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.254.148:35097
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568607354921-12282-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568607354921-12282-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.11.254.148:54335/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.131.keytab
--principal=kudu/127.11.254.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.254.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568607354921-12282-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.254.131
--webserver_port=0
--tserver_master_addrs=127.11.254.190:45247
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32289
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.254.131
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568607354921-12282-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:05:19.309875 32289 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:05:19.310148 32289 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:05:19.318806 32298 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:05:19.319617 32299 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:05:19.320346 32289 server_base.cc:1034] running on GCE node
W20260812 21:05:19.321269 32303 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:05:19.321548 32289 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:05:19.321853 32289 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:05:19.323172 32289 hybrid_clock.cc:648] HybridClock initialized: now 1786568719323164 us; error 62 us; skew 500 ppm
Aug 12 21:05:19 dist-test-slave-vpvm krb5kdc[30887](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568719, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:05:19.327334 32289 init.cc:377] Logged in from keytab as kudu/127.11.254.131@KRBTEST.COM (short username kudu)
I20260812 21:05:19.327940 32289 webserver.cc:460] Webserver started at http://127.11.254.131:41321/ using document root <none> and password file <none>
I20260812 21:05:19.328207 32289 fs_manager.cc:359] Metadata directory not provided
I20260812 21:05:19.328289 32289 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:05:19.328591 32289 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 21:05:19 dist-test-slave-vpvm krb5kdc[30887](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568719, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.131@KRBTEST.COM for HTTP/127.11.254.148@KRBTEST.COM
I20260812 21:05:19.344309 32289 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568607354921-12282-0/minicluster-data/ts-2/data/instance:
uuid: "7f9163521f0342f3b5479f138fe4976b"
format_stamp: "Formatted at 2026-08-12 21:05:19 on dist-test-slave-vpvm"
server_key: "23c0cb71672df77b31c9981f581260f5"
server_key_iv: "d79d333f57f70162b63035bf2e4cd2b8"
server_key_version: "kuduclusterkey@0"
I20260812 21:05:19.345100 32289 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568607354921-12282-0/minicluster-data/ts-2/wal/instance:
uuid: "7f9163521f0342f3b5479f138fe4976b"
format_stamp: "Formatted at 2026-08-12 21:05:19 on dist-test-slave-vpvm"
server_key: "23c0cb71672df77b31c9981f581260f5"
server_key_iv: "d79d333f57f70162b63035bf2e4cd2b8"
server_key_version: "kuduclusterkey@0"
I20260812 21:05:19.350653 32289 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.000s
I20260812 21:05:19.366176 32316 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:05:19.366443 32289 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 21:05:19.366544 32289 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568607354921-12282-0/minicluster-data/ts-2/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568607354921-12282-0/minicluster-data/ts-2/wal
uuid: "7f9163521f0342f3b5479f138fe4976b"
format_stamp: "Formatted at 2026-08-12 21:05:19 on dist-test-slave-vpvm"
server_key: "23c0cb71672df77b31c9981f581260f5"
server_key_iv: "d79d333f57f70162b63035bf2e4cd2b8"
server_key_version: "kuduclusterkey@0"
I20260812 21:05:19.366642 32289 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568607354921-12282-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568607354921-12282-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568607354921-12282-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:05:19.380041 32289 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:05:19.383209 32289 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:05:19.383550 32289 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:05:19.384084 32289 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:05:19.384598 32289 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:05:19.384689 32289 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:05:19.384760 32289 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:05:19.384817 32289 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:05:19.407965 32289 rpc_server.cc:307] RPC server started. Bound to: 127.11.254.131:43235
I20260812 21:05:19.408130 32500 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.254.131:43235 every 8 connection(s)
I20260812 21:05:19.408579 32289 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568607354921-12282-0/minicluster-data/ts-2/data/info.pb
I20260812 21:05:19.410986 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 32289
I20260812 21:05:19.411077 12282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568607354921-12282-0/minicluster-data/ts-2/wal/instance
I20260812 21:05:19.411257 12282 external_mini_cluster.cc:1428] Setting key 09eae15b4d07dd511be3b23572384adf
Aug 12 21:05:19 dist-test-slave-vpvm krb5kdc[30887](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568719, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.131@KRBTEST.COM for kudu/127.11.254.190@KRBTEST.COM
I20260812 21:05:19.422904 32038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:05:19.410068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.131:39259 (local address 127.11.254.190:45247)
0812 21:05:19.410144 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0812 21:05:19.410146 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:05:19.410327 (+ 181us) server_negotiation.cc:407] Connection header received
0812 21:05:19.412197 (+ 1870us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:05:19.412198 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:05:19.412214 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:05:19.412264 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:05:19.413441 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:05:19.414030 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:05:19.415740 (+ 1710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:05:19.416064 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:05:19.416115 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:05:19.419608 (+ 3493us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:05:19.419610 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:05:19.419610 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:05:19.419619 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:05:19.421458 (+ 1839us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:05:19.421874 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:05:19.421875 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:05:19.421875 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:05:19.421938 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:05:19.422201 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:05:19.422202 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:05:19.422202 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:05:19.422327 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:05:19.422385 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0812 21:05:19.422710 (+ 325us) server_negotiation.cc:299] Negotiation successful
0812 21:05:19.422807 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260812 21:05:19.423148 32505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:05:19.409584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:45247 (local address 127.11.254.131:39259)
0812 21:05:19.410177 (+ 593us) negotiation.cc:107] Waiting for socket to connect
0812 21:05:19.410230 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0812 21:05:19.412049 (+ 1819us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:05:19.412352 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:05:19.412357 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:05:19.412848 (+ 491us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:05:19.413350 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:05:19.413357 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:05:19.414113 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:05:19.414114 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:05:19.415621 (+ 1507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:05:19.415625 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:05:19.416222 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:05:19.416223 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:05:19.416444 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:05:19.417215 (+ 771us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:05:19.417225 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:05:19.419477 (+ 2252us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:05:19.421548 (+ 2071us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:05:19.421557 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:05:19.421560 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:05:19.421799 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:05:19.422015 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:05:19.422016 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:05:19.422016 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:05:19.422141 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:05:19.422401 (+ 260us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:05:19.422408 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:05:19.422547 (+ 139us) client_negotiation.cc:769] Sending connection context
0812 21:05:19.422662 (+ 115us) client_negotiation.cc:240] Negotiation successful
0812 21:05:19.422839 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":544,"thread_start_us":216,"threads_started":1}
I20260812 21:05:19.423799 32502 heartbeater.cc:347] Connected to a master server at 127.11.254.190:45247
I20260812 21:05:19.423919 32502 heartbeater.cc:464] Registering TS with master...
I20260812 21:05:19.424134 32502 heartbeater.cc:511] Master 127.11.254.190:45247 requested a full tablet report, sending...
I20260812 21:05:19.424513 31674 ts_manager.cc:194] Registered new tserver with Master: 7f9163521f0342f3b5479f138fe4976b (127.11.254.131:43235)
I20260812 21:05:19.424873 31674 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.254.131@KRBTEST.COM'} at 127.11.254.131:39259
I20260812 21:05:19.434679 12282 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 21:05:19.443444 32038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:05:19.436226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51904 (local address 127.11.254.190:45247)
0812 21:05:19.436302 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0812 21:05:19.436305 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 21:05:19.436537 (+ 232us) server_negotiation.cc:407] Connection header received
0812 21:05:19.436578 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:05:19.436579 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:05:19.436593 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:05:19.436673 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:05:19.437460 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:05:19.438252 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:05:19.438979 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:05:19.439263 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:05:19.439353 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:05:19.440717 (+ 1364us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:05:19.440719 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:05:19.440719 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:05:19.440726 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:05:19.442283 (+ 1557us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:05:19.442600 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:05:19.442601 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:05:19.442601 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:05:19.442670 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:05:19.442827 (+ 157us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:05:19.442828 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:05:19.442828 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:05:19.442967 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:05:19.443083 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0812 21:05:19.443247 (+ 164us) server_negotiation.cc:299] Negotiation successful
0812 21:05:19.443330 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260812 21:05:19.445139 31674 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51904:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 21:05:19.447834 31674 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 21:05:19.457827 32523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:05:19.450894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.131:43235 (local address 127.0.0.1:42482)
0812 21:05:19.451644 (+ 750us) negotiation.cc:107] Waiting for socket to connect
0812 21:05:19.451688 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 21:05:19.451776 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:05:19.452097 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:05:19.452100 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:05:19.452102 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:05:19.452180 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:05:19.452183 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:05:19.453587 (+ 1404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:05:19.453588 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:05:19.455181 (+ 1593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:05:19.455184 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:05:19.457504 (+ 2320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:05:19.457505 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:05:19.457582 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:05:19.457608 (+ 26us) client_negotiation.cc:769] Sending connection context
0812 21:05:19.457680 (+ 72us) client_negotiation.cc:240] Negotiation successful
0812 21:05:19.457741 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":687,"thread_start_us":816,"threads_started":1}
I20260812 21:05:19.458278 32520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:05:19.451207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39604 (local address 127.11.254.129:37565)
0812 21:05:19.453672 (+ 2465us) server_negotiation.cc:206] Beginning negotiation
0812 21:05:19.453676 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 21:05:19.453686 (+ 10us) server_negotiation.cc:407] Connection header received
0812 21:05:19.453719 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:05:19.453721 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:05:19.453736 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:05:19.454082 (+ 346us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:05:19.454161 (+ 79us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:05:19.455393 (+ 1232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:05:19.456726 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:05:19.457734 (+ 1008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:05:19.457790 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:05:19.457840 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0812 21:05:19.458171 (+ 331us) server_negotiation.cc:299] Negotiation successful
0812 21:05:19.458214 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2427,"thread_start_us":69,"threads_started":1}
I20260812 21:05:19.458555 32522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:05:19.451198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42482 (local address 127.11.254.131:43235)
0812 21:05:19.451534 (+ 336us) server_negotiation.cc:206] Beginning negotiation
0812 21:05:19.451540 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 21:05:19.451892 (+ 352us) server_negotiation.cc:407] Connection header received
0812 21:05:19.451975 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:05:19.451978 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:05:19.451997 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:05:19.452072 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:05:19.452245 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:05:19.453479 (+ 1234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:05:19.455407 (+ 1928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:05:19.457199 (+ 1792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:05:19.457393 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:05:19.457463 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 21:05:19.458199 (+ 736us) server_negotiation.cc:299] Negotiation successful
0812 21:05:19.458243 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":270,"thread_start_us":167,"threads_started":1}
I20260812 21:05:19.458101 32526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:05:19.450896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.129:37565 (local address 127.0.0.1:39604)
0812 21:05:19.453503 (+ 2607us) negotiation.cc:107] Waiting for socket to connect
0812 21:05:19.453515 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 21:05:19.453562 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:05:19.453892 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:05:19.453893 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:05:19.453895 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:05:19.453996 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:05:19.453998 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:05:19.455482 (+ 1484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:05:19.455484 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:05:19.456648 (+ 1164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:05:19.456651 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:05:19.457871 (+ 1220us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:05:19.457872 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:05:19.457925 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:05:19.457929 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 21:05:19.457994 (+ 65us) client_negotiation.cc:240] Negotiation successful
0812 21:05:19.458031 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2563,"mutex_wait_us":2,"thread_start_us":623,"threads_started":1}
I20260812 21:05:19.458823 32521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:05:19.451189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51516 (local address 127.11.254.130:35215)
0812 21:05:19.452124 (+ 935us) server_negotiation.cc:206] Beginning negotiation
0812 21:05:19.452128 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 21:05:19.452138 (+ 10us) server_negotiation.cc:407] Connection header received
0812 21:05:19.452205 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:05:19.452207 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:05:19.452224 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:05:19.452296 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:05:19.452425 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:05:19.453321 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:05:19.456984 (+ 3663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:05:19.458020 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:05:19.458215 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:05:19.458279 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0812 21:05:19.458648 (+ 369us) server_negotiation.cc:299] Negotiation successful
0812 21:05:19.458693 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":861,"thread_start_us":140,"threads_started":1}
I20260812 21:05:19.458568 32525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:05:19.450894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.130:35215 (local address 127.0.0.1:51516)
0812 21:05:19.451828 (+ 934us) negotiation.cc:107] Waiting for socket to connect
0812 21:05:19.451839 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 21:05:19.451884 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:05:19.452291 (+ 407us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:05:19.452292 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:05:19.452294 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:05:19.452331 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:05:19.452332 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:05:19.453398 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:05:19.453399 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:05:19.456744 (+ 3345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:05:19.456747 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:05:19.458320 (+ 1573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:05:19.458321 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:05:19.458388 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:05:19.458392 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 21:05:19.458446 (+ 54us) client_negotiation.cc:240] Negotiation successful
0812 21:05:19.458498 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":879,"mutex_wait_us":2,"thread_start_us":897,"threads_started":1}
I20260812 21:05:19.459685 31921 tablet_service.cc:1467] Processing CreateTablet for tablet f777b8b4bd39466bb0fc973bd7632627 (DEFAULT_TABLE table=test-table [id=2765d0247ff4407aaa08721793da95e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:05:19.459748 32156 tablet_service.cc:1467] Processing CreateTablet for tablet f777b8b4bd39466bb0fc973bd7632627 (DEFAULT_TABLE table=test-table [id=2765d0247ff4407aaa08721793da95e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:05:19.459750 32383 tablet_service.cc:1467] Processing CreateTablet for tablet f777b8b4bd39466bb0fc973bd7632627 (DEFAULT_TABLE table=test-table [id=2765d0247ff4407aaa08721793da95e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:05:19.460104 31921 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f777b8b4bd39466bb0fc973bd7632627. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:05:19.460171 32383 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f777b8b4bd39466bb0fc973bd7632627. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:05:19.460171 32156 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f777b8b4bd39466bb0fc973bd7632627. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:05:19.465981 32529 tablet_bootstrap.cc:492] T f777b8b4bd39466bb0fc973bd7632627 P 7f9163521f0342f3b5479f138fe4976b: Bootstrap starting.
I20260812 21:05:19.466316 32531 tablet_bootstrap.cc:492] T f777b8b4bd39466bb0fc973bd7632627 P 06aa9aeebbaf473cba89a6e94493fc93: Bootstrap starting.
I20260812 21:05:19.467935 32529 tablet_bootstrap.cc:654] T f777b8b4bd39466bb0fc973bd7632627 P 7f9163521f0342f3b5479f138fe4976b: Neither blocks nor log segments found. Creating new log.
I20260812 21:05:19.467954 32531 tablet_bootstrap.cc:654] T f777b8b4bd39466bb0fc973bd7632627 P 06aa9aeebbaf473cba89a6e94493fc93: Neither blocks nor log segments found. Creating new log.
I20260812 21:05:19.468814 32531 log.cc:824] T f777b8b4bd39466bb0fc973bd7632627 P 06aa9aeebbaf473cba89a6e94493fc93: Log is configured to *not* fsync() on all Append() calls
I20260812 21:05:19.468940 32529 log.cc:824] T f777b8b4bd39466bb0fc973bd7632627 P 7f9163521f0342f3b5479f138fe4976b: Log is configured to *not* fsync() on all Append() calls
I20260812 21:05:19.469286 32528 tablet_bootstrap.cc:492] T f777b8b4bd39466bb0fc973bd7632627 P 03bdccc09bc44ab18b1d58268e1c3936: Bootstrap starting.
I20260812 21:05:19.469867 32531 tablet_bootstrap.cc:492] T f777b8b4bd39466bb0fc973bd7632627 P 06aa9aeebbaf473cba89a6e94493fc93: No bootstrap required, opened a new log
I20260812 21:05:19.469996 32531 ts_tablet_manager.cc:1397] T f777b8b4bd39466bb0fc973bd7632627 P 06aa9aeebbaf473cba89a6e94493fc93: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260812 21:05:19.471086 32528 tablet_bootstrap.cc:654] T f777b8b4bd39466bb0fc973bd7632627 P 03bdccc09bc44ab18b1d58268e1c3936: Neither blocks nor log segments found. Creating new log.
I20260812 21:05:19.471990 32528 log.cc:824] T f777b8b4bd39466bb0fc973bd7632627 P 03bdccc09bc44ab18b1d58268e1c3936: Log is configured to *not* fsync() on all Append() calls
I20260812 21:05:19.473029 32528 tablet_bootstrap.cc:492] T f777b8b4bd39466bb0fc973bd7632627 P 03bdccc09bc44ab18b1d58268e1c3936: No bootstrap required, opened a new log
I20260812 21:05:19.473107 32528 ts_tablet_manager.cc:1397] T f777b8b4bd39466bb0fc973bd7632627 P 03bdccc09bc44ab18b1d58268e1c3936: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20260812 21:05:19.474445 32529 tablet_bootstrap.cc:492] T f777b8b4bd39466bb0fc973bd7632627 P 7f9163521f0342f3b5479f138fe4976b: No bootstrap required, opened a new log
I20260812 21:05:19.474578 32529 ts_tablet_manager.cc:1397] T f777b8b4bd39466bb0fc973bd7632627 P 7f9163521f0342f3b5479f138fe4976b: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.000s
I20260812 21:05:19.474746 32531 raft_consensus.cc:348] T f777b8b4bd39466bb0fc973bd7632627 P 06aa9aeebbaf473cba89a6e94493fc93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03bdccc09bc44ab18b1d58268e1c3936" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 37565 } } peers { permanent_uuid: "7f9163521f0342f3b5479f138fe4976b" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 43235 } } peers { permanent_uuid: "06aa9aeebbaf473cba89a6e94493fc93" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 35215 } }
I20260812 21:05:19.474951 32531 raft_consensus.cc:374] T f777b8b4bd39466bb0fc973bd7632627 P 06aa9aeebbaf473cba89a6e94493fc93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:05:19.474996 32531 raft_consensus.cc:729] T f777b8b4bd39466bb0fc973bd7632627 P 06aa9aeebbaf473cba89a6e94493fc93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06aa9aeebbaf473cba89a6e94493fc93, State: Initialized, Role: FOLLOWER
I20260812 21:05:19.475153 32531 consensus_queue.cc:260] T f777b8b4bd39466bb0fc973bd7632627 P 06aa9aeebbaf473cba89a6e94493fc93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03bdccc09bc44ab18b1d58268e1c3936" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 37565 } } peers { permanent_uuid: "7f9163521f0342f3b5479f138fe4976b" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 43235 } } peers { permanent_uuid: "06aa9aeebbaf473cba89a6e94493fc93" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 35215 } }
I20260812 21:05:19.475550 32531 ts_tablet_manager.cc:1428] T f777b8b4bd39466bb0fc973bd7632627 P 06aa9aeebbaf473cba89a6e94493fc93: Time spent starting tablet: real 0.006s user 0.001s sys 0.004s
I20260812 21:05:19.475688 32276 heartbeater.cc:503] Master 127.11.254.190:45247 was elected leader, sending a full tablet report...
I20260812 21:05:19.478128 32528 raft_consensus.cc:348] T f777b8b4bd39466bb0fc973bd7632627 P 03bdccc09bc44ab18b1d58268e1c3936 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03bdccc09bc44ab18b1d58268e1c3936" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 37565 } } peers { permanent_uuid: "7f9163521f0342f3b5479f138fe4976b" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 43235 } } peers { permanent_uuid: "06aa9aeebbaf473cba89a6e94493fc93" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 35215 } }
I20260812 21:05:19.478318 32528 raft_consensus.cc:374] T f777b8b4bd39466bb0fc973bd7632627 P 03bdccc09bc44ab18b1d58268e1c3936 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:05:19.478370 32528 raft_consensus.cc:729] T f777b8b4bd39466bb0fc973bd7632627 P 03bdccc09bc44ab18b1d58268e1c3936 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03bdccc09bc44ab18b1d58268e1c3936, State: Initialized, Role: FOLLOWER
I20260812 21:05:19.478502 32528 consensus_queue.cc:260] T f777b8b4bd39466bb0fc973bd7632627 P 03bdccc09bc44ab18b1d58268e1c3936 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03bdccc09bc44ab18b1d58268e1c3936" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 37565 } } peers { permanent_uuid: "7f9163521f0342f3b5479f138fe4976b" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 43235 } } peers { permanent_uuid: "06aa9aeebbaf473cba89a6e94493fc93" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 35215 } }
I20260812 21:05:19.478870 32528 ts_tablet_manager.cc:1428] T f777b8b4bd39466bb0fc973bd7632627 P 03bdccc09bc44ab18b1d58268e1c3936: Time spent starting tablet: real 0.006s user 0.003s sys 0.003s
I20260812 21:05:19.478960 32036 heartbeater.cc:503] Master 127.11.254.190:45247 was elected leader, sending a full tablet report...
I20260812 21:05:19.480062 32529 raft_consensus.cc:348] T f777b8b4bd39466bb0fc973bd7632627 P 7f9163521f0342f3b5479f138fe4976b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03bdccc09bc44ab18b1d58268e1c3936" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 37565 } } peers { permanent_uuid: "7f9163521f0342f3b5479f138fe4976b" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 43235 } } peers { permanent_uuid: "06aa9aeebbaf473cba89a6e94493fc93" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 35215 } }
I20260812 21:05:19.480197 32529 raft_consensus.cc:374] T f777b8b4bd39466bb0fc973bd7632627 P 7f9163521f0342f3b5479f138fe4976b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:05:19.480228 32529 raft_consensus.cc:729] T f777b8b4bd39466bb0fc973bd7632627 P 7f9163521f0342f3b5479f138fe4976b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f9163521f0342f3b5479f138fe4976b, State: Initialized, Role: FOLLOWER
I20260812 21:05:19.480340 32529 consensus_queue.cc:260] T f777b8b4bd39466bb0fc973bd7632627 P 7f9163521f0342f3b5479f138fe4976b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03bdccc09bc44ab18b1d58268e1c3936" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 37565 } } peers { permanent_uuid: "7f9163521f0342f3b5479f138fe4976b" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 43235 } } peers { permanent_uuid: "06aa9aeebbaf473cba89a6e94493fc93" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 35215 } }
I20260812 21:05:19.480609 32529 ts_tablet_manager.cc:1428] T f777b8b4bd39466bb0fc973bd7632627 P 7f9163521f0342f3b5479f138fe4976b: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260812 21:05:19.480749 32502 heartbeater.cc:503] Master 127.11.254.190:45247 was elected leader, sending a full tablet report...
W20260812 21:05:19.507597 32037 tablet.cc:2369] T f777b8b4bd39466bb0fc973bd7632627 P 03bdccc09bc44ab18b1d58268e1c3936: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 21:05:19.610222 32279 tablet.cc:2369] T f777b8b4bd39466bb0fc973bd7632627 P 06aa9aeebbaf473cba89a6e94493fc93: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 21:05:19.611752 32539 raft_consensus.cc:482] T f777b8b4bd39466bb0fc973bd7632627 P 06aa9aeebbaf473cba89a6e94493fc93 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 21:05:19.611872 32539 raft_consensus.cc:504] T f777b8b4bd39466bb0fc973bd7632627 P 06aa9aeebbaf473cba89a6e94493fc93 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03bdccc09bc44ab18b1d58268e1c3936" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 37565 } } peers { permanent_uuid: "7f9163521f0342f3b5479f138fe4976b" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 43235 } } peers { permanent_uuid: "06aa9aeebbaf473cba89a6e94493fc93" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 35215 } }
I20260812 21:05:19.612476 32539 leader_election.cc:288] T f777b8b4bd39466bb0fc973bd7632627 P 06aa9aeebbaf473cba89a6e94493fc93 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 03bdccc09bc44ab18b1d58268e1c3936 (127.11.254.129:37565), 7f9163521f0342f3b5479f138fe4976b (127.11.254.131:43235)
I20260812 21:05:19.616999 32282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:05:19.612595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.129:37565 (local address 127.11.254.130:44433)
0812 21:05:19.612767 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0812 21:05:19.612779 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 21:05:19.612826 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:05:19.613120 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:05:19.613120 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:05:19.613122 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:05:19.613351 (+ 229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:05:19.613355 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:05:19.614310 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:05:19.614311 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:05:19.615519 (+ 1208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:05:19.615523 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:05:19.616675 (+ 1152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:05:19.616676 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:05:19.616827 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:05:19.616830 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 21:05:19.616877 (+ 47us) client_negotiation.cc:240] Negotiation successful
0812 21:05:19.616923 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151}
I20260812 21:05:19.617476 32520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:05:19.612855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.130:44433 (local address 127.11.254.129:37565)
0812 21:05:19.612911 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0812 21:05:19.612913 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:05:19.612922 (+ 9us) server_negotiation.cc:407] Connection header received
0812 21:05:19.612950 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:05:19.612951 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:05:19.612961 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:05:19.613009 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:05:19.613433 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:05:19.614215 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:05:19.615730 (+ 1515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:05:19.616560 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:05:19.616632 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:05:19.616643 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0812 21:05:19.617357 (+ 714us) server_negotiation.cc:299] Negotiation successful
0812 21:05:19.617407 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260812 21:05:19.617790 31951 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f777b8b4bd39466bb0fc973bd7632627" candidate_uuid: "06aa9aeebbaf473cba89a6e94493fc93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03bdccc09bc44ab18b1d58268e1c3936" is_pre_election: true
I20260812 21:05:19.617990 31951 raft_consensus.cc:2454] T f777b8b4bd39466bb0fc973bd7632627 P 03bdccc09bc44ab18b1d58268e1c3936 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06aa9aeebbaf473cba89a6e94493fc93 in term 0.
I20260812 21:05:19.618314 32522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:05:19.613094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.130:44393 (local address 127.11.254.131:43235)
0812 21:05:19.613136 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0812 21:05:19.613138 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:05:19.613145 (+ 7us) server_negotiation.cc:407] Connection header received
0812 21:05:19.613168 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:05:19.613169 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:05:19.613177 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:05:19.613225 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:05:19.614620 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:05:19.615421 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:05:19.617222 (+ 1801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:05:19.617799 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:05:19.617849 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:05:19.617858 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 21:05:19.618196 (+ 338us) server_negotiation.cc:299] Negotiation successful
0812 21:05:19.618248 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 21:05:19.618438 32076 leader_election.cc:302] T f777b8b4bd39466bb0fc973bd7632627 P 06aa9aeebbaf473cba89a6e94493fc93 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03bdccc09bc44ab18b1d58268e1c3936, 06aa9aeebbaf473cba89a6e94493fc93; no voters:
I20260812 21:05:19.618589 32539 raft_consensus.cc:2781] T f777b8b4bd39466bb0fc973bd7632627 P 06aa9aeebbaf473cba89a6e94493fc93 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 21:05:19.618635 32539 raft_consensus.cc:482] T f777b8b4bd39466bb0fc973bd7632627 P 06aa9aeebbaf473cba89a6e94493fc93 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 21:05:19.618660 32539 raft_consensus.cc:3037] T f777b8b4bd39466bb0fc973bd7632627 P 06aa9aeebbaf473cba89a6e94493fc93 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:05:19.618770 32543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:05:19.612649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.131:43235 (local address 127.11.254.130:44393)
0812 21:05:19.612996 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0812 21:05:19.613005 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 21:05:19.613048 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:05:19.613271 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:05:19.613271 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:05:19.613273 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:05:19.614522 (+ 1249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:05:19.614525 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:05:19.615665 (+ 1140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:05:19.615666 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:05:19.617121 (+ 1455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:05:19.617125 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:05:19.617923 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:05:19.617924 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:05:19.618091 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:05:19.618098 (+ 7us) client_negotiation.cc:769] Sending connection context
0812 21:05:19.618479 (+ 381us) client_negotiation.cc:240] Negotiation successful
0812 21:05:19.618528 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":334,"thread_start_us":73,"threads_started":1}
I20260812 21:05:19.619974 32539 raft_consensus.cc:504] T f777b8b4bd39466bb0fc973bd7632627 P 06aa9aeebbaf473cba89a6e94493fc93 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03bdccc09bc44ab18b1d58268e1c3936" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 37565 } } peers { permanent_uuid: "7f9163521f0342f3b5479f138fe4976b" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 43235 } } peers { permanent_uuid: "06aa9aeebbaf473cba89a6e94493fc93" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 35215 } }
I20260812 21:05:19.620092 32539 leader_election.cc:288] T f777b8b4bd39466bb0fc973bd7632627 P 06aa9aeebbaf473cba89a6e94493fc93 [CANDIDATE]: Term 1 election: Requested vote from peers 03bdccc09bc44ab18b1d58268e1c3936 (127.11.254.129:37565), 7f9163521f0342f3b5479f138fe4976b (127.11.254.131:43235)
I20260812 21:05:19.620812 32416 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f777b8b4bd39466bb0fc973bd7632627" candidate_uuid: "06aa9aeebbaf473cba89a6e94493fc93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f9163521f0342f3b5479f138fe4976b" is_pre_election: true
I20260812 21:05:19.620980 32416 raft_consensus.cc:2454] T f777b8b4bd39466bb0fc973bd7632627 P 7f9163521f0342f3b5479f138fe4976b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06aa9aeebbaf473cba89a6e94493fc93 in term 0.
I20260812 21:05:19.621181 32414 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f777b8b4bd39466bb0fc973bd7632627" candidate_uuid: "06aa9aeebbaf473cba89a6e94493fc93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f9163521f0342f3b5479f138fe4976b"
I20260812 21:05:19.621281 32414 raft_consensus.cc:3037] T f777b8b4bd39466bb0fc973bd7632627 P 7f9163521f0342f3b5479f138fe4976b [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:05:19.622812 32414 raft_consensus.cc:2454] T f777b8b4bd39466bb0fc973bd7632627 P 7f9163521f0342f3b5479f138fe4976b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06aa9aeebbaf473cba89a6e94493fc93 in term 1.
I20260812 21:05:19.623183 32080 leader_election.cc:302] T f777b8b4bd39466bb0fc973bd7632627 P 06aa9aeebbaf473cba89a6e94493fc93 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06aa9aeebbaf473cba89a6e94493fc93, 7f9163521f0342f3b5479f138fe4976b; no voters:
I20260812 21:05:19.623157 31951 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f777b8b4bd39466bb0fc973bd7632627" candidate_uuid: "06aa9aeebbaf473cba89a6e94493fc93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03bdccc09bc44ab18b1d58268e1c3936"
I20260812 21:05:19.623268 32539 raft_consensus.cc:2781] T f777b8b4bd39466bb0fc973bd7632627 P 06aa9aeebbaf473cba89a6e94493fc93 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:05:19.623245 31951 raft_consensus.cc:3037] T f777b8b4bd39466bb0fc973bd7632627 P 03bdccc09bc44ab18b1d58268e1c3936 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:05:19.623521 32539 raft_consensus.cc:686] T f777b8b4bd39466bb0fc973bd7632627 P 06aa9aeebbaf473cba89a6e94493fc93 [term 1 LEADER]: Becoming Leader. State: Replica: 06aa9aeebbaf473cba89a6e94493fc93, State: Running, Role: LEADER
I20260812 21:05:19.623632 32539 consensus_queue.cc:237] T f777b8b4bd39466bb0fc973bd7632627 P 06aa9aeebbaf473cba89a6e94493fc93 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03bdccc09bc44ab18b1d58268e1c3936" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 37565 } } peers { permanent_uuid: "7f9163521f0342f3b5479f138fe4976b" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 43235 } } peers { permanent_uuid: "06aa9aeebbaf473cba89a6e94493fc93" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 35215 } }
I20260812 21:05:19.624724 31951 raft_consensus.cc:2454] T f777b8b4bd39466bb0fc973bd7632627 P 03bdccc09bc44ab18b1d58268e1c3936 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06aa9aeebbaf473cba89a6e94493fc93 in term 1.
I20260812 21:05:19.624755 31673 catalog_manager.cc:5495] T f777b8b4bd39466bb0fc973bd7632627 P 06aa9aeebbaf473cba89a6e94493fc93 reported cstate change: term changed from 0 to 1, leader changed from <none> to 06aa9aeebbaf473cba89a6e94493fc93 (127.11.254.130). New cstate: current_term: 1 leader_uuid: "06aa9aeebbaf473cba89a6e94493fc93" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03bdccc09bc44ab18b1d58268e1c3936" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 37565 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7f9163521f0342f3b5479f138fe4976b" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 43235 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "06aa9aeebbaf473cba89a6e94493fc93" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 35215 } health_report { overall_health: HEALTHY } } }
I20260812 21:05:19.641492 32521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:05:19.638962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51530 (local address 127.11.254.130:35215)
0812 21:05:19.639010 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0812 21:05:19.639012 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:05:19.639053 (+ 41us) server_negotiation.cc:407] Connection header received
0812 21:05:19.639152 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:05:19.639153 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:05:19.639175 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:05:19.639252 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 21:05:19.639443 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:05:19.639930 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:05:19.640688 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:05:19.640902 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:05:19.640966 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:05:19.641137 (+ 171us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 21:05:19.641248 (+ 111us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 21:05:19.641328 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0812 21:05:19.641370 (+ 42us) server_negotiation.cc:299] Negotiation successful
0812 21:05:19.641404 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 21:05:19.644199 32414 raft_consensus.cc:1261] T f777b8b4bd39466bb0fc973bd7632627 P 7f9163521f0342f3b5479f138fe4976b [term 1 FOLLOWER]: Refusing update from remote peer 06aa9aeebbaf473cba89a6e94493fc93: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 21:05:19.644601 32539 consensus_queue.cc:1035] T f777b8b4bd39466bb0fc973bd7632627 P 06aa9aeebbaf473cba89a6e94493fc93 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7f9163521f0342f3b5479f138fe4976b" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 43235 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 21:05:19.645071 31951 raft_consensus.cc:1261] T f777b8b4bd39466bb0fc973bd7632627 P 03bdccc09bc44ab18b1d58268e1c3936 [term 1 FOLLOWER]: Refusing update from remote peer 06aa9aeebbaf473cba89a6e94493fc93: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 21:05:19.645555 32539 consensus_queue.cc:1035] T f777b8b4bd39466bb0fc973bd7632627 P 06aa9aeebbaf473cba89a6e94493fc93 [LEADER]: Connected to new peer: Peer: permanent_uuid: "03bdccc09bc44ab18b1d58268e1c3936" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 37565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 21:05:19.648747 32553 mvcc.cc:205] Tried to move back new op lower bound from 7317785475659071488 to 7317785475579023360. Current Snapshot: MvccSnapshot[applied={T|T < 7317785475659071488}]
I20260812 21:05:19.649693 32554 mvcc.cc:205] Tried to move back new op lower bound from 7317785475659071488 to 7317785475579023360. Current Snapshot: MvccSnapshot[applied={T|T < 7317785475659071488}]
I20260812 21:05:19.653558 32558 mvcc.cc:205] Tried to move back new op lower bound from 7317785475659071488 to 7317785475579023360. Current Snapshot: MvccSnapshot[applied={T|T < 7317785475659071488 or (T in {7317785475659071488})}]
I20260812 21:05:19.653607 31674 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51904:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 21:05:19.653710 31674 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51904:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 21:05:19.654682 31674 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9c0b48d1b6fd40cebe4b26cd9d04d0e2: Sending DeleteTablet for 3 replicas of tablet f777b8b4bd39466bb0fc973bd7632627
I20260812 21:05:19.655313 31921 tablet_service.cc:1514] Processing DeleteTablet for tablet f777b8b4bd39466bb0fc973bd7632627 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:05:19 UTC) from {username='kudu', principal='kudu/127.11.254.190@KRBTEST.COM'} at 127.0.0.1:39604
I20260812 21:05:19.655405 32383 tablet_service.cc:1514] Processing DeleteTablet for tablet f777b8b4bd39466bb0fc973bd7632627 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:05:19 UTC) from {username='kudu', principal='kudu/127.11.254.190@KRBTEST.COM'} at 127.0.0.1:42482
I20260812 21:05:19.656010 32156 tablet_service.cc:1514] Processing DeleteTablet for tablet f777b8b4bd39466bb0fc973bd7632627 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:05:19 UTC) from {username='kudu', principal='kudu/127.11.254.190@KRBTEST.COM'} at 127.0.0.1:51516
I20260812 21:05:19.657008 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 31812
I20260812 21:05:19.683485 32575 tablet_replica.cc:330] T f777b8b4bd39466bb0fc973bd7632627 P 06aa9aeebbaf473cba89a6e94493fc93: stopping tablet replica
I20260812 21:05:19.683763 32575 raft_consensus.cc:2229] T f777b8b4bd39466bb0fc973bd7632627 P 06aa9aeebbaf473cba89a6e94493fc93 [term 1 LEADER]: Raft consensus shutting down.
I20260812 21:05:19.684038 32575 raft_consensus.cc:2258] T f777b8b4bd39466bb0fc973bd7632627 P 06aa9aeebbaf473cba89a6e94493fc93 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 21:05:19.687922 32575 ts_tablet_manager.cc:1905] T f777b8b4bd39466bb0fc973bd7632627 P 06aa9aeebbaf473cba89a6e94493fc93: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260812 21:05:19.690088 31654 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.254.129:37565 (error 108)
I20260812 21:05:19.690692 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 32048
I20260812 21:05:19.702704 32574 tablet_replica.cc:330] T f777b8b4bd39466bb0fc973bd7632627 P 7f9163521f0342f3b5479f138fe4976b: stopping tablet replica
I20260812 21:05:19.702976 32574 raft_consensus.cc:2229] T f777b8b4bd39466bb0fc973bd7632627 P 7f9163521f0342f3b5479f138fe4976b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 21:05:19.703246 32574 raft_consensus.cc:2258] T f777b8b4bd39466bb0fc973bd7632627 P 7f9163521f0342f3b5479f138fe4976b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 21:05:19.704896 32525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:05:19.704758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.129:37565 (local address 127.0.0.1:39608)
0812 21:05:19.704800 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0812 21:05:19.704856 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.129:37565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
I20260812 21:05:19.705196 32574 ts_tablet_manager.cc:1905] T f777b8b4bd39466bb0fc973bd7632627 P 7f9163521f0342f3b5479f138fe4976b: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260812 21:05:19.705227 31654 catalog_manager.cc:4584] TS 03bdccc09bc44ab18b1d58268e1c3936 (127.11.254.129:37565): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f777b8b4bd39466bb0fc973bd7632627: Network error: Client connection negotiation failed: client connection to 127.11.254.129:37565: connect: Connection refused (error 111)
I20260812 21:05:19.707052 32574 ts_tablet_manager.cc:1918] T f777b8b4bd39466bb0fc973bd7632627 P 7f9163521f0342f3b5479f138fe4976b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 21:05:19.707165 32574 log.cc:1196] T f777b8b4bd39466bb0fc973bd7632627 P 7f9163521f0342f3b5479f138fe4976b: Deleting WAL directory at /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786568607354921-12282-0/minicluster-data/ts-2/wal/wals/f777b8b4bd39466bb0fc973bd7632627
I20260812 21:05:19.707736 32574 ts_tablet_manager.cc:1939] T f777b8b4bd39466bb0fc973bd7632627 P 7f9163521f0342f3b5479f138fe4976b: Deleting consensus metadata
I20260812 21:05:19.708408 31656 catalog_manager.cc:4841] TS 7f9163521f0342f3b5479f138fe4976b (127.11.254.131:43235): tablet f777b8b4bd39466bb0fc973bd7632627 (table test-table [id=2765d0247ff4407aaa08721793da95e2]) successfully deleted
I20260812 21:05:19.710948 32525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:05:19.710820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.130:35215 (local address 127.0.0.1:51540)
0812 21:05:19.710871 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0812 21:05:19.710898 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.130:35215: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260812 21:05:19.710976 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 32289
I20260812 21:05:19.718538 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 31629
I20260812 21:05:19.730051 12282 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260812 21:05:20.078016 12282 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260812 21:05:20.078136 12282 mini_ranger.cc:66] Stopping Ranger...
I20260812 21:05:20.414049 12282 mini_ranger.cc:68] Stopped Ranger
2026-08-12 21:05:20.414 UTC [30933] LOG: received smart shutdown request
2026-08-12 21:05:20.418 UTC [30933] LOG: background worker "logical replication launcher" (PID 30954) exited with exit code 1
2026-08-12 21:05:20.418 UTC [30940] LOG: shutting down
2026-08-12 21:05:20.588 UTC [30933] LOG: database system is shut down
2026-08-12T21:05:20Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (51613 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 21:05:20 dist-test-slave-vpvm krb5kdc[32590](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 21:05:20 dist-test-slave-vpvm krb5kdc[32590](info): set up 2 sockets
Aug 12 21:05:20 dist-test-slave-vpvm krb5kdc[32590](info): commencing operation
krb5kdc: starting...
W20260812 21:05:21.914830 12282 mini_kdc.cc:121] Time spent starting KDC: real 1.212s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/krb5kdc/test-user.keytab.
Aug 12 21:05:21 dist-test-slave-vpvm krb5kdc[32590](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568721, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T21:05:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T21:05:22Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.11.254.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.11.254.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.11.254.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/krb5kdc/rangeradmin_127.11.254.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.11.254.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/krb5kdc/rangeradmin_127.11.254.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.11.254.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.11.254.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.11.254.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/krb5kdc/rangerlookup_127.11.254.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.11.254.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/krb5kdc/rangerlookup_127.11.254.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.11.254.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.11.254.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.11.254.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/krb5kdc/HTTP_127.11.254.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.11.254.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/krb5kdc/HTTP_127.11.254.148@KRBTEST.COM.keytab.
I20260812 21:05:22.092564 12282 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskymWbKN/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/minicluster-data/postgres -l logfile start
2026-08-12 21:05:24.568 UTC [32634] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-12 21:05:24.568 UTC [32634] LOG: listening on IPv4 address "127.11.254.148", port 55445
2026-08-12 21:05:24.572 UTC [32634] LOG: listening on Unix socket "/tmp/.s.PGSQL.55445"
2026-08-12 21:05:24.602 UTC [32640] LOG: database system was shut down at 2026-08-12 21:05:23 UTC
2026-08-12 21:05:24.606 UTC [32634] LOG: database system is ready to accept connections
I20260812 21:05:25.699538 12282 mini_postgres.cc:96] Postgres bound to 55445
2026-08-12 21:05:25.706 UTC [32655] FATAL: database "slave" does not exist
127.11.254.148:55445 - accepting connections
I20260812 21:05:25.707005 12282 mini_ranger.cc:161] Starting Ranger...
I20260812 21:05:25.722767 12282 mini_ranger.cc:84] Created miniranger Postgres user
I20260812 21:05:26.254431 12282 mini_ranger.cc:87] Created ranger Postgres database
I20260812 21:05:26.254515 12282 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/minicluster-data/ranger-admin
2026-08-12 21:05:26,516 [I] DB FLAVOR :POSTGRES
2026-08-12 21:05:26,516 [I] --------- Verifying Ranger DB connection ---------
2026-08-12 21:05:26,516 [I] Checking connection..
2026-08-12 21:05:26,516 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskymWbKN/build/release/bin/postgresql.jar:/tmp/dist-test-taskymWbKN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.254.148:55445/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-12 21:05:26,784 [I] Checking connection passed.
2026-08-12 21:05:26,784 [I] --------- Verifying version history table ---------
2026-08-12 21:05:26,784 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskymWbKN/build/release/bin/postgresql.jar:/tmp/dist-test-taskymWbKN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.254.148:55445/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 21:05:27,029 [I] Table x_db_version_h does not exist in database ranger
2026-08-12 21:05:27,030 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskymWbKN/build/release/bin/postgresql.jar:/tmp/dist-test-taskymWbKN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.254.148:55445/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 21:05:27,291 [I] Table x_db_version_h does not exist in database ranger
2026-08-12 21:05:27,291 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-12 21:05:27,292 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskymWbKN/build/release/bin/postgresql.jar:/tmp/dist-test-taskymWbKN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.254.148:55445/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskymWbKN/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-12 21:05:27.544 UTC [32761] WARNING: there is no transaction in progress
2026-08-12 21:05:27,553 [I] create_dbversion_catalog.sql file imported successfully
2026-08-12 21:05:27,553 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskymWbKN/build/release/bin/postgresql.jar:/tmp/dist-test-taskymWbKN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.254.148:55445/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 21:05:27,825 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-12 21:05:27,826 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-12 21:05:27,826 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskymWbKN/build/release/bin/postgresql.jar:/tmp/dist-test-taskymWbKN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.254.148:55445/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-12 21:05:28,090 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskymWbKN/build/release/bin/postgresql.jar:/tmp/dist-test-taskymWbKN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.254.148:55445/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-12 21:05:28,351 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskymWbKN/build/release/bin/postgresql.jar:/tmp/dist-test-taskymWbKN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.254.148:55445/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-vpvm.c.gcp-upstream.internal','N') ;"
2026-08-12 21:05:28,617 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskymWbKN/build/release/bin/postgresql.jar:/tmp/dist-test-taskymWbKN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.254.148:55445/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-12 21:05:28,870 [I] Table x_portal_user does not exist in database ranger
2026-08-12 21:05:28,870 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskymWbKN/build/release/bin/postgresql.jar:/tmp/dist-test-taskymWbKN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.254.148:55445/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-12 21:05:29,141 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-12 21:05:29,142 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskymWbKN/build/release/bin/postgresql.jar:/tmp/dist-test-taskymWbKN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.254.148:55445/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-12 21:05:29,413 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskymWbKN/build/release/bin/postgresql.jar:/tmp/dist-test-taskymWbKN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.254.148:55445/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-12 21:05:29,674 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-12 21:05:29,674 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskymWbKN/build/release/bin/postgresql.jar:/tmp/dist-test-taskymWbKN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.254.148:55445/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskymWbKN/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-12 21:05:30.670 UTC [448] WARNING: there is no transaction in progress
2026-08-12 21:05:30.686 UTC [448] WARNING: there is no transaction in progress
2026-08-12 21:05:30.704 UTC [448] WARNING: there is no transaction in progress
2026-08-12 21:05:30.720 UTC [448] WARNING: there is no transaction in progress
2026-08-12 21:05:30.737 UTC [448] WARNING: there is no transaction in progress
2026-08-12 21:05:30.753 UTC [448] WARNING: there is no transaction in progress
2026-08-12 21:05:30.811 UTC [448] WARNING: there is no transaction in progress
2026-08-12 21:05:30.819 UTC [448] WARNING: there is no transaction in progress
2026-08-12 21:05:30.834 UTC [448] WARNING: there is no transaction in progress
2026-08-12 21:05:30.844 UTC [448] WARNING: there is no transaction in progress
2026-08-12 21:05:30.856 UTC [448] WARNING: there is no transaction in progress
2026-08-12 21:05:30.871 UTC [448] WARNING: there is no transaction in progress
2026-08-12 21:05:30.880 UTC [448] WARNING: there is no transaction in progress
2026-08-12 21:05:30.887 UTC [448] WARNING: there is no transaction in progress
2026-08-12 21:05:30.894 UTC [448] WARNING: there is no transaction in progress
2026-08-12 21:05:30.902 UTC [448] WARNING: there is no transaction in progress
2026-08-12 21:05:31,588 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-12 21:05:31,588 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskymWbKN/build/release/bin/postgresql.jar:/tmp/dist-test-taskymWbKN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.254.148:55445/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-12 21:05:31,848 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-12 21:05:31,849 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskymWbKN/build/release/bin/postgresql.jar:/tmp/dist-test-taskymWbKN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.254.148:55445/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-12 21:05:32,095 [I] Table x_portal_user already exists in database 'ranger'
2026-08-12 21:05:32,095 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskymWbKN/build/release/bin/postgresql.jar:/tmp/dist-test-taskymWbKN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.254.148:55445/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-12 21:05:32,377 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-12 21:05:32,377 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskymWbKN/build/release/bin/postgresql.jar:/tmp/dist-test-taskymWbKN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.254.148:55445/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-12 21:05:32,636 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskymWbKN/build/release/bin/postgresql.jar:/tmp/dist-test-taskymWbKN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.254.148:55445/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-12 21:05:32,904 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskymWbKN/build/release/bin/postgresql.jar:/tmp/dist-test-taskymWbKN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.254.148:55445/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-vpvm.c.gcp-upstream.internal';"
2026-08-12 21:05:33,156 [I] CORE_DB_SCHEMA import status has been updated
2026-08-12 21:05:33,156 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskymWbKN/build/release/bin/postgresql.jar:/tmp/dist-test-taskymWbKN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.254.148:55445/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-12 21:05:33,415 [I] DB_PATCHES have already been applied
I20260812 21:05:33.421808 12282 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/minicluster-data/ranger-admin:/tmp/dist-test-taskymWbKN/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskymWbKN/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskymWbKN/build/release/bin/postgresql.jar:/tmp/dist-test-taskymWbKN/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260812 21:05:33.421877 12282 mini_ranger.cc:193] Using host: 127.11.254.148
I20260812 21:05:33.425674 12282 mini_ranger.cc:227] Ranger admin URL: http://127.11.254.148:38105
Aug 12, 2026 9:05:33 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 12, 2026 9:05:33 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 12, 2026 9:05:33 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/minicluster-data/ranger-admin/ews/webapp
Aug 12, 2026 9:05:33 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 12, 2026 9:05:33 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 12, 2026 9:05:34 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/minicluster-data/ranger-admin/ews/webapp].
Aug 12, 2026 9:05:34 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260812 21:05:34.705926 12282 mini_ranger.cc:160] Time spent starting Ranger: real 8.999s user 0.000s sys 0.011s
Aug 12, 2026 9:05:55 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 12, 2026 9:05:55 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 12, 2026 9:05:55 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 12, 2026 9:05:55 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 12, 2026 9:05:55 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 12, 2026 9:05:55 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 12, 2026 9:05:55 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 12, 2026 9:05:55 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 12, 2026 9:05:55 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 12, 2026 9:05:55 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 12, 2026 9:05:55 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 12, 2026 9:05:55 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 12, 2026 9:05:55 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 12, 2026 9:05:55 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 12, 2026 9:05:55 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 12, 2026 9:05:55 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 12, 2026 9:05:55 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 12, 2026 9:05:56 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260812 21:05:57.516103 12282 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.11.254.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.11.254.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.11.254.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/krb5kdc/rangerkms_127.11.254.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.11.254.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/krb5kdc/rangerkms_127.11.254.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.11.254.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.11.254.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.11.254.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/krb5kdc/HTTP_127.11.254.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.11.254.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/krb5kdc/HTTP_127.11.254.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 12 21:05:57 dist-test-slave-vpvm krb5kdc[32590](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568757, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260812 21:05:57.626987 12282 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260812 21:05:57.638466 12282 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260812 21:05:58.562906 12282 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260812 21:05:58.562978 12282 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/minicluster-data/ranger-kms
I20260812 21:05:58.562989 12282 mini_ranger_kms.cc:227] Using postgres at 127.11.254.148:55445
2026-08-12 21:05:58,716 [I] DB FLAVOR :POSTGRES
2026-08-12 21:05:58,716 [I] --------- Verifying Ranger DB connection ---------
2026-08-12 21:05:58,716 [I] Checking connection
2026-08-12 21:05:58,716 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskymWbKN/build/release/bin/postgresql.jar:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.254.148:55445/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-12 21:05:58,961 [I] connection success
2026-08-12 21:05:58,961 [I] --------- Verifying Ranger DB tables ---------
2026-08-12 21:05:58,961 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-12 21:05:58,962 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskymWbKN/build/release/bin/postgresql.jar:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.254.148:55445/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-12 21:05:59,208 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-12 21:05:59,208 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-12 21:05:59,208 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-12 21:05:59,208 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskymWbKN/build/release/bin/postgresql.jar:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.254.148:55445/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-12 21:05:59,476 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260812 21:05:59.949632 12282 mini_ranger_kms.cc:310] Created kms service
I20260812 21:06:00.138862 12282 mini_ranger_kms.cc:326] Created kudu user
I20260812 21:06:00.191993 12282 mini_ranger_kms.cc:343] Created rangerkms user
Aug 12 21:06:00 dist-test-slave-vpvm krb5kdc[32590](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568757, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.11.254.148@KRBTEST.COM
I20260812 21:06:00.654824 12282 mini_ranger_kms.cc:384] Added ranger policy
I20260812 21:06:00.654953 12282 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/minicluster-data/ranger-kms:/tmp/dist-test-taskymWbKN/build/release/bin/postgresql.jar:/tmp/dist-test-taskymWbKN/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskymWbKN/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260812 21:06:00.654971 12282 mini_ranger_kms.cc:240] Using host: 127.11.254.148
I20260812 21:06:00.659102 12282 mini_ranger_kms.cc:276] Ranger KMS PID: 731
I20260812 21:06:00.659224 12282 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.11.254.148:33697
Aug 12, 2026 9:06:01 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 12, 2026 9:06:01 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 12, 2026 9:06:01 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 12, 2026 9:06:01 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 12, 2026 9:06:02 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/minicluster-data/ranger-kms/ews/webapp].
Aug 12, 2026 9:06:02 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 12 21:06:02 dist-test-slave-vpvm krb5kdc[32590](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786568762, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.11.254.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:06:03.260166 12282 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.633s user 0.002s sys 0.136s
I20260812 21:06:03.260275 12282 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260812 21:06:03.260324 12282 mini_ranger_kms.cc:401] 127.11.254.148:33697/kms/v1/keys
Aug 12 21:06:03 dist-test-slave-vpvm krb5kdc[32590](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568762, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.11.254.148@KRBTEST.COM for HTTP/127.11.254.148@KRBTEST.COM
Aug 12 21:06:03 dist-test-slave-vpvm krb5kdc[32590](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568762, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.11.254.148@KRBTEST.COM for HTTP/127.11.254.148@KRBTEST.COM
Aug 12 21:06:06 dist-test-slave-vpvm krb5kdc[32590](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568766, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.11.254.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.254.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.254.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.190.keytab.
Entry for principal kudu/127.11.254.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.190.keytab.
I20260812 21:06:06.229254 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.254.190:40171
--webserver_interface=127.11.254.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.190.keytab
--principal=kudu/127.11.254.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.254.148:46733
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.254.190:40171
--ranger_config_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.254.148:33697/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:06:06.319331 811 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:06:06.319571 811 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:06:06.319594 811 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:06:06.320824 811 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 21:06:06.320871 811 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:06:06.320885 811 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:06:06.320899 811 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 21:06:06.320921 811 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 21:06:06.322569 811 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.254.148:46733
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/minicluster-data/master-0/wal
--ranger_kms_url=127.11.254.148:33697/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.11.254.190:40171
--ranger_config_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.190.keytab
--principal=kudu/127.11.254.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.254.190:40171
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.254.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.811
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:06:06.322813 811 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:06:06.323056 811 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:06:06.328354 818 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:06:06.328349 824 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:06:06.328569 820 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:06:06.328565 811 server_base.cc:1034] running on GCE node
I20260812 21:06:06.328907 811 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:06:06.329212 811 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:06:06.330359 811 hybrid_clock.cc:648] HybridClock initialized: now 1786568766330337 us; error 38 us; skew 500 ppm
Aug 12 21:06:06 dist-test-slave-vpvm krb5kdc[32590](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568766, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:06:06.332875 811 init.cc:377] Logged in from keytab as kudu/127.11.254.190@KRBTEST.COM (short username kudu)
I20260812 21:06:06.333365 811 webserver.cc:460] Webserver started at http://127.11.254.190:42689/ using document root <none> and password file <none>
I20260812 21:06:06.333571 811 fs_manager.cc:359] Metadata directory not provided
I20260812 21:06:06.333608 811 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:06:06.333704 811 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 21:06:06 dist-test-slave-vpvm krb5kdc[32590](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568766, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.190@KRBTEST.COM for HTTP/127.11.254.148@KRBTEST.COM
I20260812 21:06:06.390285 811 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/minicluster-data/master-0/data/instance:
uuid: "ca4c639e0fef454a830a71a98575fb7a"
format_stamp: "Formatted at 2026-08-12 21:06:06 on dist-test-slave-vpvm"
server_key: "2785df0f1c644aa3a3af3ece4581acc6"
server_key_iv: "bd821f11f2c24d56ea3d6ec5ee12cec4"
server_key_version: "kuduclusterkey@0"
I20260812 21:06:06.390700 811 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/minicluster-data/master-0/wal/instance:
uuid: "ca4c639e0fef454a830a71a98575fb7a"
format_stamp: "Formatted at 2026-08-12 21:06:06 on dist-test-slave-vpvm"
server_key: "2785df0f1c644aa3a3af3ece4581acc6"
server_key_iv: "bd821f11f2c24d56ea3d6ec5ee12cec4"
server_key_version: "kuduclusterkey@0"
I20260812 21:06:06.392174 811 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 21:06:06.407838 839 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:06:06.408159 811 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 21:06:06.408268 811 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/minicluster-data/master-0/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/minicluster-data/master-0/wal
uuid: "ca4c639e0fef454a830a71a98575fb7a"
format_stamp: "Formatted at 2026-08-12 21:06:06 on dist-test-slave-vpvm"
server_key: "2785df0f1c644aa3a3af3ece4581acc6"
server_key_iv: "bd821f11f2c24d56ea3d6ec5ee12cec4"
server_key_version: "kuduclusterkey@0"
I20260812 21:06:06.408367 811 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:06:06.436355 811 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:06:06.438431 811 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:06:06.438592 811 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:06:06.442631 811 rpc_server.cc:307] RPC server started. Bound to: 127.11.254.190:40171
I20260812 21:06:06.442705 924 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.254.190:40171 every 8 connection(s)
I20260812 21:06:06.443044 811 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
I20260812 21:06:06.443804 925 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:06:06.446166 925 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca4c639e0fef454a830a71a98575fb7a: Bootstrap starting.
I20260812 21:06:06.446769 925 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ca4c639e0fef454a830a71a98575fb7a: Neither blocks nor log segments found. Creating new log.
I20260812 21:06:06.446918 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 811
I20260812 21:06:06.447041 12282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/minicluster-data/master-0/wal/instance
I20260812 21:06:06.447119 925 log.cc:824] T 00000000000000000000000000000000 P ca4c639e0fef454a830a71a98575fb7a: Log is configured to *not* fsync() on all Append() calls
I20260812 21:06:06.447261 12282 external_mini_cluster.cc:1428] Setting key 0daff525364e6089898514e46fab86ec
I20260812 21:06:06.447978 925 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca4c639e0fef454a830a71a98575fb7a: No bootstrap required, opened a new log
I20260812 21:06:06.450668 925 raft_consensus.cc:348] T 00000000000000000000000000000000 P ca4c639e0fef454a830a71a98575fb7a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca4c639e0fef454a830a71a98575fb7a" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 40171 } }
I20260812 21:06:06.450863 925 raft_consensus.cc:374] T 00000000000000000000000000000000 P ca4c639e0fef454a830a71a98575fb7a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:06:06.450896 925 raft_consensus.cc:729] T 00000000000000000000000000000000 P ca4c639e0fef454a830a71a98575fb7a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca4c639e0fef454a830a71a98575fb7a, State: Initialized, Role: FOLLOWER
I20260812 21:06:06.451030 925 consensus_queue.cc:260] T 00000000000000000000000000000000 P ca4c639e0fef454a830a71a98575fb7a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca4c639e0fef454a830a71a98575fb7a" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 40171 } }
I20260812 21:06:06.451122 925 raft_consensus.cc:388] T 00000000000000000000000000000000 P ca4c639e0fef454a830a71a98575fb7a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 21:06:06.451162 925 raft_consensus.cc:482] T 00000000000000000000000000000000 P ca4c639e0fef454a830a71a98575fb7a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 21:06:06.451211 925 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ca4c639e0fef454a830a71a98575fb7a [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:06:06.451807 925 raft_consensus.cc:504] T 00000000000000000000000000000000 P ca4c639e0fef454a830a71a98575fb7a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca4c639e0fef454a830a71a98575fb7a" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 40171 } }
I20260812 21:06:06.451924 925 leader_election.cc:302] T 00000000000000000000000000000000 P ca4c639e0fef454a830a71a98575fb7a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ca4c639e0fef454a830a71a98575fb7a; no voters:
I20260812 21:06:06.452104 925 leader_election.cc:288] T 00000000000000000000000000000000 P ca4c639e0fef454a830a71a98575fb7a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 21:06:06.452169 930 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ca4c639e0fef454a830a71a98575fb7a [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:06:06.452355 930 raft_consensus.cc:686] T 00000000000000000000000000000000 P ca4c639e0fef454a830a71a98575fb7a [term 1 LEADER]: Becoming Leader. State: Replica: ca4c639e0fef454a830a71a98575fb7a, State: Running, Role: LEADER
I20260812 21:06:06.452399 925 sys_catalog.cc:564] T 00000000000000000000000000000000 P ca4c639e0fef454a830a71a98575fb7a [sys.catalog]: configured and running, proceeding with master startup.
I20260812 21:06:06.452505 930 consensus_queue.cc:237] T 00000000000000000000000000000000 P ca4c639e0fef454a830a71a98575fb7a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca4c639e0fef454a830a71a98575fb7a" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 40171 } }
I20260812 21:06:06.453100 933 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca4c639e0fef454a830a71a98575fb7a [sys.catalog]: SysCatalogTable state changed. Reason: New leader ca4c639e0fef454a830a71a98575fb7a. Latest consensus state: current_term: 1 leader_uuid: "ca4c639e0fef454a830a71a98575fb7a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca4c639e0fef454a830a71a98575fb7a" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 40171 } } }
I20260812 21:06:06.453207 933 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca4c639e0fef454a830a71a98575fb7a [sys.catalog]: This master's current role is: LEADER
I20260812 21:06:06.454030 932 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca4c639e0fef454a830a71a98575fb7a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ca4c639e0fef454a830a71a98575fb7a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca4c639e0fef454a830a71a98575fb7a" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 40171 } } }
I20260812 21:06:06.454107 932 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca4c639e0fef454a830a71a98575fb7a [sys.catalog]: This master's current role is: LEADER
I20260812 21:06:06.454265 925 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260812 21:06:06.454294 947 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Aug 12 21:06:06 dist-test-slave-vpvm krb5kdc[32590](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568766, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.254.190@KRBTEST.COM
I20260812 21:06:06.457233 947 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 21:06:06.460026 947 catalog_manager.cc:1347] Generated new cluster ID: 9e8eccbda05c4baa97b72d0902300dbd
I20260812 21:06:06.460119 947 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 21:06:06.461088 929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:06.447842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51554 (local address 127.11.254.190:40171)
0812 21:06:06.448025 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0812 21:06:06.448036 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 21:06:06.448072 (+ 36us) server_negotiation.cc:407] Connection header received
0812 21:06:06.448575 (+ 503us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:06:06.448585 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:06:06.448673 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:06:06.448837 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:06:06.449489 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:06:06.450940 (+ 1451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:06:06.451831 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:06:06.452499 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:06:06.454065 (+ 1566us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:06:06.456136 (+ 2071us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:06:06.456177 (+ 41us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:06:06.456177 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:06:06.456282 (+ 105us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:06:06.459349 (+ 3067us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:06:06.459770 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:06:06.459772 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:06:06.459773 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:06:06.459877 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:06:06.460094 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:06:06.460095 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:06:06.460095 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:06:06.460438 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:06:06.460588 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0812 21:06:06.460723 (+ 135us) server_negotiation.cc:299] Negotiation successful
0812 21:06:06.460882 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":96,"threads_started":1}
I20260812 21:06:06.477900 947 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 21:06:06.478798 947 catalog_manager.cc:1504] Loading token signing keys...
I20260812 21:06:06.495429 947 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ca4c639e0fef454a830a71a98575fb7a: Generated new TSK 0
I20260812 21:06:06.495682 947 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.254.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.254.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.254.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.129.keytab.
Entry for principal kudu/127.11.254.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.129.keytab.
I20260812 21:06:07.154011 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.254.148:33697/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.254.129:0
--local_ip_for_outbound_sockets=127.11.254.129
--webserver_interface=127.11.254.129
--webserver_port=0
--tserver_master_addrs=127.11.254.190:40171
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.129.keytab
--principal=kudu/127.11.254.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.254.148:46733
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:06:07.244580 997 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:06:07.244885 997 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:06:07.244931 997 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:06:07.246703 997 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:06:07.246778 997 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:06:07.246819 997 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.254.129
I20260812 21:06:07.249112 997 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.254.148:46733
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.11.254.148:33697/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.129.keytab
--principal=kudu/127.11.254.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.254.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.254.129
--webserver_port=0
--tserver_master_addrs=127.11.254.190:40171
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.997
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.254.129
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:06:07.249390 997 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:06:07.249706 997 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:06:07.255147 1011 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:06:07.255306 1009 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:06:07.255473 997 server_base.cc:1034] running on GCE node
W20260812 21:06:07.255162 1008 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:06:07.255817 997 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:06:07.256093 997 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:06:07.257263 997 hybrid_clock.cc:648] HybridClock initialized: now 1786568767257248 us; error 34 us; skew 500 ppm
Aug 12 21:06:07 dist-test-slave-vpvm krb5kdc[32590](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568767, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:06:07.259922 997 init.cc:377] Logged in from keytab as kudu/127.11.254.129@KRBTEST.COM (short username kudu)
I20260812 21:06:07.260401 997 webserver.cc:460] Webserver started at http://127.11.254.129:36467/ using document root <none> and password file <none>
I20260812 21:06:07.260643 997 fs_manager.cc:359] Metadata directory not provided
I20260812 21:06:07.260746 997 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:06:07.260900 997 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 21:06:07 dist-test-slave-vpvm krb5kdc[32590](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568767, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.129@KRBTEST.COM for HTTP/127.11.254.148@KRBTEST.COM
I20260812 21:06:07.281126 997 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/minicluster-data/ts-0/data/instance:
uuid: "5248b7f95db2494c8ef3d1bad13ac18b"
format_stamp: "Formatted at 2026-08-12 21:06:07 on dist-test-slave-vpvm"
server_key: "830fd5870d8563cf85a5467892fcc915"
server_key_iv: "a66263bad77eae71d6339437a21e298b"
server_key_version: "kuduclusterkey@0"
I20260812 21:06:07.281697 997 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/minicluster-data/ts-0/wal/instance:
uuid: "5248b7f95db2494c8ef3d1bad13ac18b"
format_stamp: "Formatted at 2026-08-12 21:06:07 on dist-test-slave-vpvm"
server_key: "830fd5870d8563cf85a5467892fcc915"
server_key_iv: "a66263bad77eae71d6339437a21e298b"
server_key_version: "kuduclusterkey@0"
I20260812 21:06:07.283224 997 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 21:06:07.305859 1023 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:06:07.306119 997 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 21:06:07.306233 997 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/minicluster-data/ts-0/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/minicluster-data/ts-0/wal
uuid: "5248b7f95db2494c8ef3d1bad13ac18b"
format_stamp: "Formatted at 2026-08-12 21:06:07 on dist-test-slave-vpvm"
server_key: "830fd5870d8563cf85a5467892fcc915"
server_key_iv: "a66263bad77eae71d6339437a21e298b"
server_key_version: "kuduclusterkey@0"
I20260812 21:06:07.306341 997 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:06:07.330976 997 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:06:07.333542 997 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:06:07.333768 997 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:06:07.334156 997 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:06:07.334689 997 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:06:07.334792 997 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:06:07.334896 997 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:06:07.334970 997 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:06:07.342007 997 rpc_server.cc:307] RPC server started. Bound to: 127.11.254.129:33855
I20260812 21:06:07.342175 1202 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.254.129:33855 every 8 connection(s)
I20260812 21:06:07.342568 997 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/minicluster-data/ts-0/data/info.pb
I20260812 21:06:07.351619 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 997
I20260812 21:06:07.351743 12282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/minicluster-data/ts-0/wal/instance
I20260812 21:06:07.351923 12282 external_mini_cluster.cc:1428] Setting key a925ffad27af49e5af8f6c52b8d6e33f
Aug 12 21:06:07 dist-test-slave-vpvm krb5kdc[32590](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568767, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.129@KRBTEST.COM for kudu/127.11.254.190@KRBTEST.COM
I20260812 21:06:07.357419 1207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:07.343452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.129:36175 (local address 127.11.254.190:40171)
0812 21:06:07.347063 (+ 3611us) server_negotiation.cc:206] Beginning negotiation
0812 21:06:07.347066 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 21:06:07.347075 (+ 9us) server_negotiation.cc:407] Connection header received
0812 21:06:07.347104 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:06:07.347106 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:06:07.347132 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:06:07.347213 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:06:07.348026 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:06:07.348844 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:06:07.350388 (+ 1544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:06:07.350713 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:06:07.350792 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:06:07.354122 (+ 3330us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:06:07.354123 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:06:07.354124 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:06:07.354131 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:06:07.355925 (+ 1794us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:06:07.356435 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:06:07.356436 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:06:07.356437 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:06:07.356497 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:06:07.356754 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:06:07.356755 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:06:07.356756 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:06:07.356877 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:06:07.356932 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0812 21:06:07.357227 (+ 295us) server_negotiation.cc:299] Negotiation successful
0812 21:06:07.357317 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3575,"thread_start_us":103,"threads_started":1}
I20260812 21:06:07.357560 1206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:07.343411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:40171 (local address 127.11.254.129:36175)
0812 21:06:07.346089 (+ 2678us) negotiation.cc:107] Waiting for socket to connect
0812 21:06:07.346153 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0812 21:06:07.346843 (+ 690us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:06:07.347333 (+ 490us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:06:07.347338 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:06:07.347620 (+ 282us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:06:07.347904 (+ 284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:06:07.347910 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:06:07.348951 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:06:07.348952 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:06:07.350290 (+ 1338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:06:07.350294 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:06:07.350825 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:06:07.350826 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:06:07.350988 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:06:07.352278 (+ 1290us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:06:07.352289 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:06:07.354003 (+ 1714us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:06:07.356022 (+ 2019us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:06:07.356029 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:06:07.356032 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:06:07.356337 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:06:07.356570 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:06:07.356571 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:06:07.356571 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:06:07.356682 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:06:07.356944 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:06:07.356950 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:06:07.357093 (+ 143us) client_negotiation.cc:769] Sending connection context
0812 21:06:07.357180 (+ 87us) client_negotiation.cc:240] Negotiation successful
0812 21:06:07.357334 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2609,"thread_start_us":91,"threads_started":1}
I20260812 21:06:07.358460 1204 heartbeater.cc:347] Connected to a master server at 127.11.254.190:40171
I20260812 21:06:07.358587 1204 heartbeater.cc:464] Registering TS with master...
I20260812 21:06:07.358807 1204 heartbeater.cc:511] Master 127.11.254.190:40171 requested a full tablet report, sending...
I20260812 21:06:07.359426 867 ts_manager.cc:194] Registered new tserver with Master: 5248b7f95db2494c8ef3d1bad13ac18b (127.11.254.129:33855)
I20260812 21:06:07.360191 867 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.254.129@KRBTEST.COM'} at 127.11.254.129:36175
WARNING: no policy specified for kudu/127.11.254.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.254.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.254.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.130.keytab.
Entry for principal kudu/127.11.254.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.130.keytab.
I20260812 21:06:07.385411 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.254.148:33697/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.254.130:0
--local_ip_for_outbound_sockets=127.11.254.130
--webserver_interface=127.11.254.130
--webserver_port=0
--tserver_master_addrs=127.11.254.190:40171
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.130.keytab
--principal=kudu/127.11.254.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.254.148:46733
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 21:06:07 dist-test-slave-vpvm krb5kdc[32590](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786568767, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260812 21:06:07.508296 1211 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:06:07.508693 1211 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:06:07.508774 1211 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:06:07.510646 1211 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:06:07.510784 1211 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:06:07.510874 1211 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.254.130
I20260812 21:06:07.513233 1211 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.254.148:46733
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.11.254.148:33697/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.130.keytab
--principal=kudu/127.11.254.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.254.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.254.130
--webserver_port=0
--tserver_master_addrs=127.11.254.190:40171
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1211
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.254.130
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:06:07.513720 1211 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:06:07.514093 1211 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:06:07.520999 1228 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:06:07.521256 1225 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:06:07.521090 1223 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:06:07.521709 1211 server_base.cc:1034] running on GCE node
I20260812 21:06:07.521984 1211 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:06:07.522284 1211 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:06:07.529796 1211 hybrid_clock.cc:648] HybridClock initialized: now 1786568767529758 us; error 43 us; skew 500 ppm
Aug 12 21:06:07 dist-test-slave-vpvm krb5kdc[32590](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568767, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:06:07.533334 1211 init.cc:377] Logged in from keytab as kudu/127.11.254.130@KRBTEST.COM (short username kudu)
I20260812 21:06:07.533903 1211 webserver.cc:460] Webserver started at http://127.11.254.130:34183/ using document root <none> and password file <none>
I20260812 21:06:07.534197 1211 fs_manager.cc:359] Metadata directory not provided
I20260812 21:06:07.534257 1211 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:06:07.534437 1211 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 21:06:07 dist-test-slave-vpvm krb5kdc[32590](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568767, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.130@KRBTEST.COM for HTTP/127.11.254.148@KRBTEST.COM
I20260812 21:06:07.547670 1211 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/minicluster-data/ts-1/data/instance:
uuid: "f7fe70f4546147589958c6297ea17bda"
format_stamp: "Formatted at 2026-08-12 21:06:07 on dist-test-slave-vpvm"
server_key: "ee69a2994ee36b7e120b890db08df776"
server_key_iv: "70a5fcd50af05adaa84c6fe05e1612ed"
server_key_version: "kuduclusterkey@0"
I20260812 21:06:07.548120 1211 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/minicluster-data/ts-1/wal/instance:
uuid: "f7fe70f4546147589958c6297ea17bda"
format_stamp: "Formatted at 2026-08-12 21:06:07 on dist-test-slave-vpvm"
server_key: "ee69a2994ee36b7e120b890db08df776"
server_key_iv: "70a5fcd50af05adaa84c6fe05e1612ed"
server_key_version: "kuduclusterkey@0"
I20260812 21:06:07.549844 1211 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260812 21:06:07.573318 1245 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:06:07.573580 1211 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 21:06:07.573705 1211 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/minicluster-data/ts-1/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/minicluster-data/ts-1/wal
uuid: "f7fe70f4546147589958c6297ea17bda"
format_stamp: "Formatted at 2026-08-12 21:06:07 on dist-test-slave-vpvm"
server_key: "ee69a2994ee36b7e120b890db08df776"
server_key_iv: "70a5fcd50af05adaa84c6fe05e1612ed"
server_key_version: "kuduclusterkey@0"
I20260812 21:06:07.573800 1211 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:06:07.601105 1211 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:06:07.603662 1211 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:06:07.603828 1211 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:06:07.604056 1211 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:06:07.604492 1211 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:06:07.604548 1211 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:06:07.604578 1211 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:06:07.604637 1211 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:06:07.612262 1211 rpc_server.cc:307] RPC server started. Bound to: 127.11.254.130:37341
I20260812 21:06:07.612768 1211 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/minicluster-data/ts-1/data/info.pb
I20260812 21:06:07.617885 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 1211
I20260812 21:06:07.618027 12282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/minicluster-data/ts-1/wal/instance
I20260812 21:06:07.618242 12282 external_mini_cluster.cc:1428] Setting key c44388b364c941543821a3279aa7dd5c
I20260812 21:06:07.622066 1407 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.254.130:37341 every 8 connection(s)
Aug 12 21:06:07 dist-test-slave-vpvm krb5kdc[32590](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568767, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.130@KRBTEST.COM for kudu/127.11.254.190@KRBTEST.COM
I20260812 21:06:07.629690 1207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:07.613602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.130:54411 (local address 127.11.254.190:40171)
0812 21:06:07.613662 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0812 21:06:07.613664 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:06:07.613926 (+ 262us) server_negotiation.cc:407] Connection header received
0812 21:06:07.614722 (+ 796us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:06:07.614723 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:06:07.614741 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:06:07.614772 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:06:07.615676 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:06:07.616441 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:06:07.621255 (+ 4814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:06:07.621573 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:06:07.621643 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:06:07.626321 (+ 4678us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:06:07.626323 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:06:07.626324 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:06:07.626332 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:06:07.628026 (+ 1694us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:06:07.628539 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:06:07.628541 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:06:07.628541 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:06:07.628615 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:06:07.628897 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:06:07.628898 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:06:07.628899 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:06:07.629049 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:06:07.629100 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0812 21:06:07.629476 (+ 376us) server_negotiation.cc:299] Negotiation successful
0812 21:06:07.629574 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
WARNING: no policy specified for kudu/127.11.254.131@KRBTEST.COM; defaulting to no policy
I20260812 21:06:07.630563 1411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:07.613589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:40171 (local address 127.11.254.130:54411)
0812 21:06:07.613776 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0812 21:06:07.613829 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0812 21:06:07.614576 (+ 747us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:06:07.614969 (+ 393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:06:07.614974 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:06:07.615228 (+ 254us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:06:07.615537 (+ 309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:06:07.615545 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:06:07.616569 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:06:07.616569 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:06:07.621132 (+ 4563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:06:07.621141 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:06:07.621771 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:06:07.621772 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:06:07.621944 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:06:07.624424 (+ 2480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:06:07.624435 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:06:07.626183 (+ 1748us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:06:07.628124 (+ 1941us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:06:07.628133 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:06:07.628136 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:06:07.628447 (+ 311us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:06:07.628682 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:06:07.628683 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:06:07.628683 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:06:07.628822 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:06:07.629135 (+ 313us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:06:07.629141 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:06:07.629322 (+ 181us) client_negotiation.cc:769] Sending connection context
0812 21:06:07.630130 (+ 808us) client_negotiation.cc:240] Negotiation successful
0812 21:06:07.630315 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":93,"threads_started":1}
I20260812 21:06:07.631188 1408 heartbeater.cc:347] Connected to a master server at 127.11.254.190:40171
I20260812 21:06:07.631307 1408 heartbeater.cc:464] Registering TS with master...
I20260812 21:06:07.631531 1408 heartbeater.cc:511] Master 127.11.254.190:40171 requested a full tablet report, sending...
I20260812 21:06:07.631960 867 ts_manager.cc:194] Registered new tserver with Master: f7fe70f4546147589958c6297ea17bda (127.11.254.130:37341)
I20260812 21:06:07.632472 867 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.254.130@KRBTEST.COM'} at 127.11.254.130:54411
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.254.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.254.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.131.keytab.
Entry for principal kudu/127.11.254.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.131.keytab.
I20260812 21:06:07.658780 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.254.148:33697/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.254.131:0
--local_ip_for_outbound_sockets=127.11.254.131
--webserver_interface=127.11.254.131
--webserver_port=0
--tserver_master_addrs=127.11.254.190:40171
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.131.keytab
--principal=kudu/127.11.254.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.254.148:46733
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:06:07.755868 1414 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:06:07.756165 1414 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:06:07.756198 1414 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:06:07.757803 1414 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:06:07.757869 1414 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:06:07.757910 1414 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.254.131
I20260812 21:06:07.759893 1414 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.254.148:46733
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.11.254.148:33697/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.131.keytab
--principal=kudu/127.11.254.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.254.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.254.131
--webserver_port=0
--tserver_master_addrs=127.11.254.190:40171
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1414
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.254.131
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:06:07.760166 1414 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:06:07.760432 1414 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:06:07.766144 1425 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:06:07.766189 1424 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:06:07.766651 1429 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:06:07.767307 1414 server_base.cc:1034] running on GCE node
I20260812 21:06:07.767567 1414 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:06:07.767858 1414 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:06:07.769042 1414 hybrid_clock.cc:648] HybridClock initialized: now 1786568767769013 us; error 41 us; skew 500 ppm
Aug 12 21:06:07 dist-test-slave-vpvm krb5kdc[32590](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568767, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:06:07.772460 1414 init.cc:377] Logged in from keytab as kudu/127.11.254.131@KRBTEST.COM (short username kudu)
I20260812 21:06:07.773075 1414 webserver.cc:460] Webserver started at http://127.11.254.131:41833/ using document root <none> and password file <none>
I20260812 21:06:07.773365 1414 fs_manager.cc:359] Metadata directory not provided
I20260812 21:06:07.773437 1414 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:06:07.773592 1414 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 21:06:07 dist-test-slave-vpvm krb5kdc[32590](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568767, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.131@KRBTEST.COM for HTTP/127.11.254.148@KRBTEST.COM
I20260812 21:06:07.786545 1414 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/minicluster-data/ts-2/data/instance:
uuid: "80cb37a528d743bb9fc081e1010b5fe3"
format_stamp: "Formatted at 2026-08-12 21:06:07 on dist-test-slave-vpvm"
server_key: "ac07403b2fa8d0a4deacac29cbc17b15"
server_key_iv: "0568f1047ff541d2508540d530556189"
server_key_version: "kuduclusterkey@0"
I20260812 21:06:07.786993 1414 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/minicluster-data/ts-2/wal/instance:
uuid: "80cb37a528d743bb9fc081e1010b5fe3"
format_stamp: "Formatted at 2026-08-12 21:06:07 on dist-test-slave-vpvm"
server_key: "ac07403b2fa8d0a4deacac29cbc17b15"
server_key_iv: "0568f1047ff541d2508540d530556189"
server_key_version: "kuduclusterkey@0"
I20260812 21:06:07.794620 1414 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.002s sys 0.000s
I20260812 21:06:07.809288 1440 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:06:07.809581 1414 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 21:06:07.809711 1414 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/minicluster-data/ts-2/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/minicluster-data/ts-2/wal
uuid: "80cb37a528d743bb9fc081e1010b5fe3"
format_stamp: "Formatted at 2026-08-12 21:06:07 on dist-test-slave-vpvm"
server_key: "ac07403b2fa8d0a4deacac29cbc17b15"
server_key_iv: "0568f1047ff541d2508540d530556189"
server_key_version: "kuduclusterkey@0"
I20260812 21:06:07.809844 1414 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:06:07.823405 1414 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:06:07.825387 1414 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:06:07.825536 1414 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:06:07.825906 1414 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:06:07.826375 1414 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:06:07.826424 1414 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:06:07.826457 1414 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:06:07.826474 1414 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:06:07.842208 1414 rpc_server.cc:307] RPC server started. Bound to: 127.11.254.131:45447
I20260812 21:06:07.842401 1615 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.254.131:45447 every 8 connection(s)
I20260812 21:06:07.842710 1414 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/minicluster-data/ts-2/data/info.pb
I20260812 21:06:07.849656 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 1414
I20260812 21:06:07.849767 12282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/minicluster-data/ts-2/wal/instance
I20260812 21:06:07.849985 12282 external_mini_cluster.cc:1428] Setting key 862d6a110582fa8ef4868603e1eb513f
Aug 12 21:06:07 dist-test-slave-vpvm krb5kdc[32590](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568767, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.131@KRBTEST.COM for kudu/127.11.254.190@KRBTEST.COM
I20260812 21:06:07.855158 1207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:07.843707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.131:45829 (local address 127.11.254.190:40171)
0812 21:06:07.843766 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0812 21:06:07.843769 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 21:06:07.844182 (+ 413us) server_negotiation.cc:407] Connection header received
0812 21:06:07.845094 (+ 912us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:06:07.845095 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:06:07.845110 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:06:07.845205 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:06:07.846292 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:06:07.846958 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:06:07.848370 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:06:07.848650 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:06:07.848689 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:06:07.851626 (+ 2937us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:06:07.851628 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:06:07.851628 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:06:07.851637 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:06:07.853349 (+ 1712us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:06:07.853933 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:06:07.853935 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:06:07.853935 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:06:07.854034 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:06:07.854330 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:06:07.854331 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:06:07.854331 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:06:07.854493 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:06:07.854546 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0812 21:06:07.854965 (+ 419us) server_negotiation.cc:299] Negotiation successful
0812 21:06:07.855059 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260812 21:06:07.855983 1619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:07.843672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:40171 (local address 127.11.254.131:45829)
0812 21:06:07.844018 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0812 21:06:07.844064 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 21:06:07.844978 (+ 914us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:06:07.845261 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:06:07.845265 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:06:07.845753 (+ 488us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:06:07.846183 (+ 430us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:06:07.846191 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:06:07.847082 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:06:07.847083 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:06:07.848266 (+ 1183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:06:07.848269 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:06:07.848802 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:06:07.848803 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:06:07.848943 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:06:07.849529 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:06:07.849540 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:06:07.851514 (+ 1974us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:06:07.853447 (+ 1933us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:06:07.853456 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:06:07.853458 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:06:07.853844 (+ 386us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:06:07.854111 (+ 267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:06:07.854111 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:06:07.854112 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:06:07.854257 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:06:07.854595 (+ 338us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:06:07.854602 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:06:07.854791 (+ 189us) client_negotiation.cc:769] Sending connection context
0812 21:06:07.855552 (+ 761us) client_negotiation.cc:240] Negotiation successful
0812 21:06:07.855742 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":188,"threads_started":1}
I20260812 21:06:07.856768 1617 heartbeater.cc:347] Connected to a master server at 127.11.254.190:40171
I20260812 21:06:07.856881 1617 heartbeater.cc:464] Registering TS with master...
I20260812 21:06:07.857075 1617 heartbeater.cc:511] Master 127.11.254.190:40171 requested a full tablet report, sending...
I20260812 21:06:07.857550 867 ts_manager.cc:194] Registered new tserver with Master: 80cb37a528d743bb9fc081e1010b5fe3 (127.11.254.131:45447)
I20260812 21:06:07.858064 867 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.254.131@KRBTEST.COM'} at 127.11.254.131:45829
I20260812 21:06:07.867436 12282 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 21:06:07.868201 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 997
I20260812 21:06:07.883850 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 1211
I20260812 21:06:07.897318 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 1414
I20260812 21:06:07.914703 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 811
I20260812 21:06:07.931391 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.254.190:40171
--webserver_interface=127.11.254.190
--webserver_port=42689
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.190.keytab
--principal=kudu/127.11.254.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.254.148:46733
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.254.190:40171
--ranger_config_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.254.148:33697/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.11.254.148:33697/kms with env {KRB5CCNAME=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:06:08.016150 1620 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:06:08.016386 1620 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:06:08.016417 1620 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:06:08.017661 1620 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 21:06:08.017712 1620 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:06:08.017726 1620 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:06:08.017736 1620 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 21:06:08.017748 1620 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 21:06:08.019551 1620 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.254.148:46733
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.11.254.148:33697/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.11.254.190:40171
--ranger_config_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.190.keytab
--principal=kudu/127.11.254.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.254.190:40171
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.254.190
--webserver_port=42689
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1620
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:06:08.019783 1620 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:06:08.019984 1620 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:06:08.025300 1628 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:06:08.025311 1631 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:06:08.025321 1626 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:06:08.025666 1620 server_base.cc:1034] running on GCE node
I20260812 21:06:08.025918 1620 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:06:08.026254 1620 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:06:08.027436 1620 hybrid_clock.cc:648] HybridClock initialized: now 1786568768027418 us; error 25 us; skew 500 ppm
Aug 12 21:06:08 dist-test-slave-vpvm krb5kdc[32590](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568768, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:06:08.030058 1620 init.cc:377] Logged in from keytab as kudu/127.11.254.190@KRBTEST.COM (short username kudu)
I20260812 21:06:08.030579 1620 webserver.cc:460] Webserver started at http://127.11.254.190:42689/ using document root <none> and password file <none>
I20260812 21:06:08.030812 1620 fs_manager.cc:359] Metadata directory not provided
I20260812 21:06:08.030869 1620 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 12 21:06:08 dist-test-slave-vpvm krb5kdc[32590](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568768, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.190@KRBTEST.COM for HTTP/127.11.254.148@KRBTEST.COM
I20260812 21:06:08.105122 1620 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 21:06:08.106052 1640 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:06:08.106231 1620 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 21:06:08.106336 1620 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/minicluster-data/master-0/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/minicluster-data/master-0/wal
uuid: "ca4c639e0fef454a830a71a98575fb7a"
format_stamp: "Formatted at 2026-08-12 21:06:06 on dist-test-slave-vpvm"
server_key: "2785df0f1c644aa3a3af3ece4581acc6"
server_key_iv: "bd821f11f2c24d56ea3d6ec5ee12cec4"
server_key_version: "kuduclusterkey@0"
I20260812 21:06:08.106736 1620 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:06:08.116024 1620 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:06:08.118114 1620 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:06:08.118273 1620 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:06:08.123425 1620 rpc_server.cc:307] RPC server started. Bound to: 127.11.254.190:40171
I20260812 21:06:08.123500 1732 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.254.190:40171 every 8 connection(s)
I20260812 21:06:08.123980 1620 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
I20260812 21:06:08.125418 1733 sys_catalog.cc:263] Verifying existing consensus state
I20260812 21:06:08.126187 1733 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca4c639e0fef454a830a71a98575fb7a: Bootstrap starting.
I20260812 21:06:08.128916 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 1620
I20260812 21:06:08.129220 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.254.148:33697/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.254.129:33855
--local_ip_for_outbound_sockets=127.11.254.129
--tserver_master_addrs=127.11.254.190:40171
--webserver_port=36467
--webserver_interface=127.11.254.129
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.129.keytab
--principal=kudu/127.11.254.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.254.148:46733
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.11.254.148:33697/kms with env {KRB5CCNAME=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 21:06:08.178004 1733 log.cc:824] T 00000000000000000000000000000000 P ca4c639e0fef454a830a71a98575fb7a: Log is configured to *not* fsync() on all Append() calls
I20260812 21:06:08.180277 1733 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca4c639e0fef454a830a71a98575fb7a: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 21:06:08.180570 1733 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca4c639e0fef454a830a71a98575fb7a: Bootstrap complete.
I20260812 21:06:08.183044 1733 raft_consensus.cc:348] T 00000000000000000000000000000000 P ca4c639e0fef454a830a71a98575fb7a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca4c639e0fef454a830a71a98575fb7a" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 40171 } }
I20260812 21:06:08.183315 1733 raft_consensus.cc:729] T 00000000000000000000000000000000 P ca4c639e0fef454a830a71a98575fb7a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca4c639e0fef454a830a71a98575fb7a, State: Initialized, Role: FOLLOWER
I20260812 21:06:08.183494 1733 consensus_queue.cc:260] T 00000000000000000000000000000000 P ca4c639e0fef454a830a71a98575fb7a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca4c639e0fef454a830a71a98575fb7a" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 40171 } }
I20260812 21:06:08.183571 1733 raft_consensus.cc:388] T 00000000000000000000000000000000 P ca4c639e0fef454a830a71a98575fb7a [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 21:06:08.183621 1733 raft_consensus.cc:482] T 00000000000000000000000000000000 P ca4c639e0fef454a830a71a98575fb7a [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 21:06:08.183665 1733 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ca4c639e0fef454a830a71a98575fb7a [term 1 FOLLOWER]: Advancing to term 2
I20260812 21:06:08.184698 1733 raft_consensus.cc:504] T 00000000000000000000000000000000 P ca4c639e0fef454a830a71a98575fb7a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca4c639e0fef454a830a71a98575fb7a" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 40171 } }
I20260812 21:06:08.184799 1733 leader_election.cc:302] T 00000000000000000000000000000000 P ca4c639e0fef454a830a71a98575fb7a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ca4c639e0fef454a830a71a98575fb7a; no voters:
I20260812 21:06:08.184979 1733 leader_election.cc:288] T 00000000000000000000000000000000 P ca4c639e0fef454a830a71a98575fb7a [CANDIDATE]: Term 2 election: Requested vote from peers
I20260812 21:06:08.185070 1740 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ca4c639e0fef454a830a71a98575fb7a [term 2 FOLLOWER]: Leader election won for term 2
I20260812 21:06:08.185271 1733 sys_catalog.cc:564] T 00000000000000000000000000000000 P ca4c639e0fef454a830a71a98575fb7a [sys.catalog]: configured and running, proceeding with master startup.
I20260812 21:06:08.185289 1740 raft_consensus.cc:686] T 00000000000000000000000000000000 P ca4c639e0fef454a830a71a98575fb7a [term 2 LEADER]: Becoming Leader. State: Replica: ca4c639e0fef454a830a71a98575fb7a, State: Running, Role: LEADER
I20260812 21:06:08.185385 1740 consensus_queue.cc:237] T 00000000000000000000000000000000 P ca4c639e0fef454a830a71a98575fb7a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca4c639e0fef454a830a71a98575fb7a" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 40171 } }
I20260812 21:06:08.185668 1740 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca4c639e0fef454a830a71a98575fb7a [sys.catalog]: SysCatalogTable state changed. Reason: New leader ca4c639e0fef454a830a71a98575fb7a. Latest consensus state: current_term: 2 leader_uuid: "ca4c639e0fef454a830a71a98575fb7a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca4c639e0fef454a830a71a98575fb7a" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 40171 } } }
I20260812 21:06:08.185738 1740 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca4c639e0fef454a830a71a98575fb7a [sys.catalog]: This master's current role is: LEADER
I20260812 21:06:08.185878 1741 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca4c639e0fef454a830a71a98575fb7a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "ca4c639e0fef454a830a71a98575fb7a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca4c639e0fef454a830a71a98575fb7a" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 40171 } } }
I20260812 21:06:08.185941 1741 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca4c639e0fef454a830a71a98575fb7a [sys.catalog]: This master's current role is: LEADER
I20260812 21:06:08.186201 1748 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 21:06:08.186682 1748 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 21:06:08.186815 1748 catalog_manager.cc:1259] Loaded cluster ID: 9e8eccbda05c4baa97b72d0902300dbd
I20260812 21:06:08.186849 1748 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 21:06:08.187059 1733 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260812 21:06:08.190798 1748 catalog_manager.cc:1504] Loading token signing keys...
I20260812 21:06:08.191411 1748 catalog_manager.cc:5879] T 00000000000000000000000000000000 P ca4c639e0fef454a830a71a98575fb7a: Loaded TSK: 0
I20260812 21:06:08.191742 1748 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 21:06:08.258646 1736 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:06:08.258985 1736 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:06:08.259042 1736 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:06:08.261032 1736 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:06:08.261108 1736 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:06:08.261153 1736 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.254.129
I20260812 21:06:08.263959 1736 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.254.148:46733
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.11.254.148:33697/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.129.keytab
--principal=kudu/127.11.254.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.254.129:33855
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.254.129
--webserver_port=36467
--tserver_master_addrs=127.11.254.190:40171
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1736
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.254.129
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:06:08.264305 1736 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:06:08.264632 1736 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:06:08.273711 1783 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:06:08.274016 1782 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:06:08.274186 1786 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:06:08.274359 1736 server_base.cc:1034] running on GCE node
I20260812 21:06:08.274585 1736 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:06:08.274857 1736 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:06:08.276008 1736 hybrid_clock.cc:648] HybridClock initialized: now 1786568768275978 us; error 46 us; skew 500 ppm
Aug 12 21:06:08 dist-test-slave-vpvm krb5kdc[32590](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568768, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:06:08.280699 1736 init.cc:377] Logged in from keytab as kudu/127.11.254.129@KRBTEST.COM (short username kudu)
I20260812 21:06:08.281275 1736 webserver.cc:460] Webserver started at http://127.11.254.129:36467/ using document root <none> and password file <none>
I20260812 21:06:08.281564 1736 fs_manager.cc:359] Metadata directory not provided
I20260812 21:06:08.281661 1736 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 12 21:06:08 dist-test-slave-vpvm krb5kdc[32590](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568768, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.129@KRBTEST.COM for HTTP/127.11.254.148@KRBTEST.COM
I20260812 21:06:08.300477 1736 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.000s
I20260812 21:06:08.303121 1801 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:06:08.304397 1736 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 21:06:08.304527 1736 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/minicluster-data/ts-0/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/minicluster-data/ts-0/wal
uuid: "5248b7f95db2494c8ef3d1bad13ac18b"
format_stamp: "Formatted at 2026-08-12 21:06:07 on dist-test-slave-vpvm"
server_key: "830fd5870d8563cf85a5467892fcc915"
server_key_iv: "a66263bad77eae71d6339437a21e298b"
server_key_version: "kuduclusterkey@0"
I20260812 21:06:08.305227 1736 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:06:08.333197 1736 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:06:08.336227 1736 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:06:08.336493 1736 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:06:08.337013 1736 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:06:08.337575 1736 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:06:08.337627 1736 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:06:08.337661 1736 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:06:08.337688 1736 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:06:08.347139 1736 rpc_server.cc:307] RPC server started. Bound to: 127.11.254.129:33855
I20260812 21:06:08.347675 1736 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/minicluster-data/ts-0/data/info.pb
I20260812 21:06:08.350407 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 1736
I20260812 21:06:08.350901 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.254.148:33697/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.254.130:37341
--local_ip_for_outbound_sockets=127.11.254.130
--tserver_master_addrs=127.11.254.190:40171
--webserver_port=34183
--webserver_interface=127.11.254.130
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.130.keytab
--principal=kudu/127.11.254.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.254.148:46733
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.11.254.148:33697/kms with env {KRB5CCNAME=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 21:06:08.358640 1985 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.254.129:33855 every 8 connection(s)
Aug 12 21:06:08 dist-test-slave-vpvm krb5kdc[32590](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568768, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.129@KRBTEST.COM for kudu/127.11.254.190@KRBTEST.COM
I20260812 21:06:08.387094 1990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:08.359520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:40171 (local address 127.11.254.129:59067)
0812 21:06:08.359960 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0812 21:06:08.360012 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0812 21:06:08.360653 (+ 641us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:06:08.362075 (+ 1422us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:06:08.362079 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:06:08.362333 (+ 254us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:06:08.362642 (+ 309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:06:08.362649 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:06:08.366700 (+ 4051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:06:08.366702 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:06:08.368085 (+ 1383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:06:08.368089 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:06:08.369462 (+ 1373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:06:08.369463 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:06:08.369606 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:06:08.370300 (+ 694us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:06:08.370322 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:06:08.372751 (+ 2429us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:06:08.379310 (+ 6559us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:06:08.379324 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:06:08.379326 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:06:08.380526 (+ 1200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:06:08.385357 (+ 4831us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:06:08.385358 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:06:08.385359 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:06:08.385576 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:06:08.386134 (+ 558us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:06:08.386142 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:06:08.386395 (+ 253us) client_negotiation.cc:769] Sending connection context
0812 21:06:08.386492 (+ 97us) client_negotiation.cc:240] Negotiation successful
0812 21:06:08.386828 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":381,"thread_start_us":114,"threads_started":1}
I20260812 21:06:08.388151 1991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:08.359606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.129:59067 (local address 127.11.254.190:40171)
0812 21:06:08.361401 (+ 1795us) server_negotiation.cc:206] Beginning negotiation
0812 21:06:08.361414 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 21:06:08.361464 (+ 50us) server_negotiation.cc:407] Connection header received
0812 21:06:08.361638 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:06:08.361645 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:06:08.361705 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:06:08.361849 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:06:08.362791 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:06:08.366529 (+ 3738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:06:08.368202 (+ 1673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:06:08.368889 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:06:08.369016 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:06:08.372924 (+ 3908us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:06:08.372930 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:06:08.372930 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:06:08.372959 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:06:08.379173 (+ 6214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:06:08.385113 (+ 5940us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:06:08.385118 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:06:08.385118 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:06:08.385274 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:06:08.385651 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:06:08.385653 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:06:08.385653 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:06:08.386019 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:06:08.386103 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0812 21:06:08.387787 (+ 1684us) server_negotiation.cc:299] Negotiation successful
0812 21:06:08.387932 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1534,"thread_start_us":86,"threads_started":1}
I20260812 21:06:08.389430 1986 heartbeater.cc:347] Connected to a master server at 127.11.254.190:40171
I20260812 21:06:08.389528 1986 heartbeater.cc:464] Registering TS with master...
I20260812 21:06:08.389716 1986 heartbeater.cc:511] Master 127.11.254.190:40171 requested a full tablet report, sending...
W20260812 21:06:08.390694 1986 heartbeater.cc:644] Failed to heartbeat to 127.11.254.190:40171 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260812 21:06:08.390765 1986 heartbeater.cc:464] Registering TS with master...
I20260812 21:06:08.390827 1986 heartbeater.cc:511] Master 127.11.254.190:40171 requested a full tablet report, sending...
I20260812 21:06:08.391253 1986 heartbeater.cc:464] Registering TS with master...
I20260812 21:06:08.391330 1986 heartbeater.cc:511] Master 127.11.254.190:40171 requested a full tablet report, sending...
W20260812 21:06:08.391639 1986 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 21:06:08.457826 1989 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:06:08.458163 1989 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:06:08.458201 1989 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:06:08.459985 1989 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:06:08.460065 1989 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:06:08.460110 1989 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.254.130
I20260812 21:06:08.461858 1989 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.254.148:46733
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.11.254.148:33697/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.130.keytab
--principal=kudu/127.11.254.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.254.130:37341
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.254.130
--webserver_port=34183
--tserver_master_addrs=127.11.254.190:40171
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1989
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.254.130
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:06:08.462162 1989 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:06:08.462420 1989 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:06:08.467625 2007 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:06:08.467680 2003 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:06:08.467972 2004 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:06:08.469419 1989 server_base.cc:1034] running on GCE node
I20260812 21:06:08.469777 1989 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:06:08.470108 1989 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:06:08.477507 1989 hybrid_clock.cc:648] HybridClock initialized: now 1786568768477372 us; error 142 us; skew 500 ppm
Aug 12 21:06:08 dist-test-slave-vpvm krb5kdc[32590](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568768, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:06:08.480261 1989 init.cc:377] Logged in from keytab as kudu/127.11.254.130@KRBTEST.COM (short username kudu)
I20260812 21:06:08.480806 1989 webserver.cc:460] Webserver started at http://127.11.254.130:34183/ using document root <none> and password file <none>
I20260812 21:06:08.481035 1989 fs_manager.cc:359] Metadata directory not provided
I20260812 21:06:08.481105 1989 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 12 21:06:08 dist-test-slave-vpvm krb5kdc[32590](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568768, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.130@KRBTEST.COM for HTTP/127.11.254.148@KRBTEST.COM
I20260812 21:06:08.503921 1989 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 21:06:08.506488 2024 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:06:08.506712 1989 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 21:06:08.506816 1989 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/minicluster-data/ts-1/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/minicluster-data/ts-1/wal
uuid: "f7fe70f4546147589958c6297ea17bda"
format_stamp: "Formatted at 2026-08-12 21:06:07 on dist-test-slave-vpvm"
server_key: "ee69a2994ee36b7e120b890db08df776"
server_key_iv: "70a5fcd50af05adaa84c6fe05e1612ed"
server_key_version: "kuduclusterkey@0"
I20260812 21:06:08.507357 1989 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:06:08.528744 1989 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:06:08.531253 1989 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:06:08.531415 1989 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:06:08.531690 1989 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:06:08.532094 1989 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:06:08.532148 1989 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:06:08.532184 1989 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:06:08.532209 1989 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:06:08.539852 1989 rpc_server.cc:307] RPC server started. Bound to: 127.11.254.130:37341
I20260812 21:06:08.540362 1989 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/minicluster-data/ts-1/data/info.pb
I20260812 21:06:08.542128 2215 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.254.130:37341 every 8 connection(s)
Aug 12 21:06:08 dist-test-slave-vpvm krb5kdc[32590](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568768, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.130@KRBTEST.COM for kudu/127.11.254.190@KRBTEST.COM
I20260812 21:06:08.548686 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 1989
I20260812 21:06:08.549010 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.254.148:33697/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.254.131:45447
--local_ip_for_outbound_sockets=127.11.254.131
--tserver_master_addrs=127.11.254.190:40171
--webserver_port=41833
--webserver_interface=127.11.254.131
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.131.keytab
--principal=kudu/127.11.254.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.254.148:46733
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.11.254.148:33697/kms with env {KRB5CCNAME=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 21:06:08.552371 1991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:08.542689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.130:55547 (local address 127.11.254.190:40171)
0812 21:06:08.542752 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0812 21:06:08.542754 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:06:08.542768 (+ 14us) server_negotiation.cc:407] Connection header received
0812 21:06:08.543218 (+ 450us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:06:08.543220 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:06:08.543233 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:06:08.543318 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:06:08.543881 (+ 563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:06:08.544659 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:06:08.545743 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:06:08.546135 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:06:08.546209 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:06:08.548641 (+ 2432us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:06:08.548643 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:06:08.548644 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:06:08.548655 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:06:08.550508 (+ 1853us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:06:08.551121 (+ 613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:06:08.551123 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:06:08.551123 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:06:08.551194 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:06:08.551515 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:06:08.551516 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:06:08.551517 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:06:08.551688 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:06:08.551741 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0812 21:06:08.552170 (+ 429us) server_negotiation.cc:299] Negotiation successful
0812 21:06:08.552272 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260812 21:06:08.552987 2218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:08.542366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:40171 (local address 127.11.254.130:55547)
0812 21:06:08.542554 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0812 21:06:08.542600 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 21:06:08.543087 (+ 487us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:06:08.543379 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:06:08.543383 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:06:08.543568 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:06:08.543774 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:06:08.543779 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:06:08.544781 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:06:08.544782 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:06:08.545645 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:06:08.545648 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:06:08.546272 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:06:08.546274 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:06:08.546400 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:06:08.547010 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:06:08.547020 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:06:08.548443 (+ 1423us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:06:08.550633 (+ 2190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:06:08.550642 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:06:08.550645 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:06:08.551019 (+ 374us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:06:08.551269 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:06:08.551270 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:06:08.551271 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:06:08.551426 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:06:08.551794 (+ 368us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:06:08.551801 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:06:08.551990 (+ 189us) client_negotiation.cc:769] Sending connection context
0812 21:06:08.552521 (+ 531us) client_negotiation.cc:240] Negotiation successful
0812 21:06:08.552734 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":91,"threads_started":1}
I20260812 21:06:08.553483 2216 heartbeater.cc:347] Connected to a master server at 127.11.254.190:40171
I20260812 21:06:08.553607 2216 heartbeater.cc:464] Registering TS with master...
I20260812 21:06:08.553826 2216 heartbeater.cc:511] Master 127.11.254.190:40171 requested a full tablet report, sending...
W20260812 21:06:08.554499 2216 heartbeater.cc:644] Failed to heartbeat to 127.11.254.190:40171 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260812 21:06:08.554555 2216 heartbeater.cc:464] Registering TS with master...
I20260812 21:06:08.554603 2216 heartbeater.cc:511] Master 127.11.254.190:40171 requested a full tablet report, sending...
I20260812 21:06:08.555009 2216 heartbeater.cc:464] Registering TS with master...
I20260812 21:06:08.555099 2216 heartbeater.cc:511] Master 127.11.254.190:40171 requested a full tablet report, sending...
W20260812 21:06:08.555542 2216 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 21:06:08.644631 2219 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:06:08.644850 2219 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:06:08.644883 2219 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:06:08.646145 2219 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:06:08.646193 2219 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:06:08.646232 2219 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.254.131
I20260812 21:06:08.647892 2219 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.254.148:46733
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.11.254.148:33697/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.131.keytab
--principal=kudu/127.11.254.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.254.131:45447
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.254.131
--webserver_port=41833
--tserver_master_addrs=127.11.254.190:40171
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2219
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.254.131
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:06:08.648159 2219 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:06:08.648444 2219 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:06:08.653591 2229 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:06:08.654117 2227 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:06:08.654729 2225 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:06:08.655560 2219 server_base.cc:1034] running on GCE node
I20260812 21:06:08.655856 2219 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:06:08.656144 2219 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:06:08.657320 2219 hybrid_clock.cc:648] HybridClock initialized: now 1786568768657301 us; error 31 us; skew 500 ppm
Aug 12 21:06:08 dist-test-slave-vpvm krb5kdc[32590](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568768, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:06:08.660055 2219 init.cc:377] Logged in from keytab as kudu/127.11.254.131@KRBTEST.COM (short username kudu)
I20260812 21:06:08.660563 2219 webserver.cc:460] Webserver started at http://127.11.254.131:41833/ using document root <none> and password file <none>
I20260812 21:06:08.660817 2219 fs_manager.cc:359] Metadata directory not provided
I20260812 21:06:08.660888 2219 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 12 21:06:08 dist-test-slave-vpvm krb5kdc[32590](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568768, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.131@KRBTEST.COM for HTTP/127.11.254.148@KRBTEST.COM
I20260812 21:06:08.677206 2219 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 21:06:08.683287 2239 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:06:08.683555 2219 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.000s sys 0.000s
I20260812 21:06:08.683681 2219 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/minicluster-data/ts-2/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/minicluster-data/ts-2/wal
uuid: "80cb37a528d743bb9fc081e1010b5fe3"
format_stamp: "Formatted at 2026-08-12 21:06:07 on dist-test-slave-vpvm"
server_key: "ac07403b2fa8d0a4deacac29cbc17b15"
server_key_iv: "0568f1047ff541d2508540d530556189"
server_key_version: "kuduclusterkey@0"
I20260812 21:06:08.685158 2219 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:06:08.701814 2219 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:06:08.704609 2219 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:06:08.704775 2219 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:06:08.705071 2219 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:06:08.705488 2219 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:06:08.705542 2219 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:06:08.705585 2219 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:06:08.705603 2219 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:06:08.713645 2219 rpc_server.cc:307] RPC server started. Bound to: 127.11.254.131:45447
I20260812 21:06:08.713762 2412 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.254.131:45447 every 8 connection(s)
I20260812 21:06:08.714198 2219 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/minicluster-data/ts-2/data/info.pb
I20260812 21:06:08.716279 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 2219
Aug 12 21:06:08 dist-test-slave-vpvm krb5kdc[32590](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568768, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.131@KRBTEST.COM for kudu/127.11.254.190@KRBTEST.COM
I20260812 21:06:08.724997 1991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:08.715051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.131:55471 (local address 127.11.254.190:40171)
0812 21:06:08.715125 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0812 21:06:08.715127 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:06:08.715358 (+ 231us) server_negotiation.cc:407] Connection header received
0812 21:06:08.716028 (+ 670us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:06:08.716029 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:06:08.716041 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:06:08.716094 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:06:08.716993 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:06:08.717757 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:06:08.718894 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:06:08.719109 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:06:08.719169 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:06:08.721813 (+ 2644us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:06:08.721815 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:06:08.721815 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:06:08.721823 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:06:08.723417 (+ 1594us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:06:08.723914 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:06:08.723916 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:06:08.723916 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:06:08.723983 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:06:08.724226 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:06:08.724227 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:06:08.724227 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:06:08.724387 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:06:08.724444 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0812 21:06:08.724803 (+ 359us) server_negotiation.cc:299] Negotiation successful
0812 21:06:08.724889 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260812 21:06:08.725489 2417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:08.715014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:40171 (local address 127.11.254.131:55471)
0812 21:06:08.715207 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0812 21:06:08.715258 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0812 21:06:08.715915 (+ 657us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:06:08.716173 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:06:08.716178 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:06:08.716421 (+ 243us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:06:08.716895 (+ 474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:06:08.716903 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:06:08.717875 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:06:08.717876 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:06:08.718805 (+ 929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:06:08.718808 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:06:08.719218 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:06:08.719219 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:06:08.719366 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:06:08.720148 (+ 782us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:06:08.720159 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:06:08.721720 (+ 1561us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:06:08.723530 (+ 1810us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:06:08.723538 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:06:08.723540 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:06:08.723843 (+ 303us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:06:08.724045 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:06:08.724046 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:06:08.724046 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:06:08.724168 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:06:08.724451 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:06:08.724457 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:06:08.724640 (+ 183us) client_negotiation.cc:769] Sending connection context
0812 21:06:08.724758 (+ 118us) client_negotiation.cc:240] Negotiation successful
0812 21:06:08.724920 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":81,"threads_started":1}
I20260812 21:06:08.726747 2415 heartbeater.cc:347] Connected to a master server at 127.11.254.190:40171
I20260812 21:06:08.726861 2415 heartbeater.cc:464] Registering TS with master...
I20260812 21:06:08.727090 2415 heartbeater.cc:511] Master 127.11.254.190:40171 requested a full tablet report, sending...
W20260812 21:06:08.727957 2415 heartbeater.cc:644] Failed to heartbeat to 127.11.254.190:40171 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260812 21:06:08.728031 2415 heartbeater.cc:464] Registering TS with master...
I20260812 21:06:08.728101 2415 heartbeater.cc:511] Master 127.11.254.190:40171 requested a full tablet report, sending...
I20260812 21:06:08.728428 2415 heartbeater.cc:464] Registering TS with master...
I20260812 21:06:08.728506 2415 heartbeater.cc:511] Master 127.11.254.190:40171 requested a full tablet report, sending...
W20260812 21:06:08.729157 2415 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260812 21:06:08.729255 2420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:08.716653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60928 (local address 127.11.254.190:40171)
0812 21:06:08.717045 (+ 392us) server_negotiation.cc:206] Beginning negotiation
0812 21:06:08.717047 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:06:08.717055 (+ 8us) server_negotiation.cc:407] Connection header received
0812 21:06:08.717077 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:06:08.717078 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:06:08.717089 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:06:08.717158 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:06:08.718449 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:06:08.719229 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:06:08.721526 (+ 2297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:06:08.721866 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:06:08.721936 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:06:08.724203 (+ 2267us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:06:08.724205 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:06:08.724205 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:06:08.724213 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:06:08.727348 (+ 3135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:06:08.727819 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:06:08.727820 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:06:08.727820 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:06:08.727890 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:06:08.728426 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:06:08.728427 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:06:08.728428 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:06:08.728661 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:06:08.728830 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0812 21:06:08.728937 (+ 107us) server_negotiation.cc:299] Negotiation successful
0812 21:06:08.729031 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":359,"thread_start_us":85,"threads_started":1}
Aug 12 21:06:09 dist-test-slave-vpvm krb5kdc[32590](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786568769, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:06:09.391814 1986 heartbeater.cc:464] Registering TS with master...
I20260812 21:06:09.391991 1986 heartbeater.cc:511] Master 127.11.254.190:40171 requested a full tablet report, sending...
I20260812 21:06:09.392448 1669 ts_manager.cc:194] Registered new tserver with Master: 5248b7f95db2494c8ef3d1bad13ac18b (127.11.254.129:33855)
I20260812 21:06:09.393504 1669 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.254.129@KRBTEST.COM'} at 127.11.254.129:59067
I20260812 21:06:09.555773 2216 heartbeater.cc:464] Registering TS with master...
I20260812 21:06:09.555984 2216 heartbeater.cc:511] Master 127.11.254.190:40171 requested a full tablet report, sending...
I20260812 21:06:09.556371 1669 ts_manager.cc:194] Registered new tserver with Master: f7fe70f4546147589958c6297ea17bda (127.11.254.130:37341)
I20260812 21:06:09.556875 1669 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.254.130@KRBTEST.COM'} at 127.11.254.130:55547
I20260812 21:06:09.729357 2415 heartbeater.cc:464] Registering TS with master...
I20260812 21:06:09.729521 2415 heartbeater.cc:511] Master 127.11.254.190:40171 requested a full tablet report, sending...
I20260812 21:06:09.729892 1669 ts_manager.cc:194] Registered new tserver with Master: 80cb37a528d743bb9fc081e1010b5fe3 (127.11.254.131:45447)
I20260812 21:06:09.730314 12282 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 21:06:09.730502 1669 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.254.131@KRBTEST.COM'} at 127.11.254.131:55471
Aug 12 21:06:09 dist-test-slave-vpvm krb5kdc[32590](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568769, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.190@KRBTEST.COM for HTTP/127.11.254.148@KRBTEST.COM
I20260812 21:06:09.747442 2446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:09.731361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60930 (local address 127.11.254.190:40171)
0812 21:06:09.739024 (+ 7663us) server_negotiation.cc:206] Beginning negotiation
0812 21:06:09.739027 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 21:06:09.739040 (+ 13us) server_negotiation.cc:407] Connection header received
0812 21:06:09.739074 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:06:09.739076 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:06:09.739091 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:06:09.739184 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:06:09.740448 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:06:09.741243 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:06:09.742314 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:06:09.742631 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:06:09.742699 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:06:09.744191 (+ 1492us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:06:09.744192 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:06:09.744193 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:06:09.744200 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:06:09.745879 (+ 1679us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:06:09.746293 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:06:09.746294 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:06:09.746294 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:06:09.746361 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:06:09.746562 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:06:09.746563 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:06:09.746563 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:06:09.746705 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:06:09.746797 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0812 21:06:09.747247 (+ 450us) server_negotiation.cc:299] Negotiation successful
0812 21:06:09.747335 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7629,"thread_start_us":76,"threads_started":1}
I20260812 21:06:09.749809 1669 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60930:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 21:06:09.752223 1669 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 21:06:09.765362 2460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:09.759872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48956 (local address 127.11.254.129:33855)
0812 21:06:09.760027 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0812 21:06:09.760031 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 21:06:09.760418 (+ 387us) server_negotiation.cc:407] Connection header received
0812 21:06:09.760574 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:06:09.760577 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:06:09.760596 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:06:09.760652 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:06:09.761111 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:06:09.762277 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:06:09.764024 (+ 1747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:06:09.764792 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:06:09.764863 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:06:09.764929 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0812 21:06:09.765219 (+ 290us) server_negotiation.cc:299] Negotiation successful
0812 21:06:09.765267 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":68,"threads_started":1}
I20260812 21:06:09.765631 2458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:09.759382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.129:33855 (local address 127.0.0.1:48956)
0812 21:06:09.760314 (+ 932us) negotiation.cc:107] Waiting for socket to connect
0812 21:06:09.760349 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0812 21:06:09.760459 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:06:09.760779 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:06:09.760781 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:06:09.760787 (+ 6us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:06:09.761022 (+ 235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:06:09.761024 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:06:09.762394 (+ 1370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:06:09.762395 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:06:09.763907 (+ 1512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:06:09.763910 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:06:09.764980 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:06:09.764981 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:06:09.765079 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:06:09.765094 (+ 15us) client_negotiation.cc:769] Sending connection context
0812 21:06:09.765482 (+ 388us) client_negotiation.cc:240] Negotiation successful
0812 21:06:09.765558 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":913,"mutex_wait_us":166,"thread_start_us":112,"threads_started":1}
I20260812 21:06:09.766726 2462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:09.759371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.131:45447 (local address 127.0.0.1:42266)
0812 21:06:09.760294 (+ 923us) negotiation.cc:107] Waiting for socket to connect
0812 21:06:09.760339 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 21:06:09.760458 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:06:09.760778 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:06:09.760781 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:06:09.760783 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:06:09.760883 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:06:09.760885 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:06:09.762353 (+ 1468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:06:09.762353 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:06:09.765050 (+ 2697us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:06:09.765053 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:06:09.766436 (+ 1383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:06:09.766437 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:06:09.766514 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:06:09.766518 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 21:06:09.766603 (+ 85us) client_negotiation.cc:240] Negotiation successful
0812 21:06:09.766646 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":885,"mutex_wait_us":1472,"thread_start_us":93,"threads_started":1}
I20260812 21:06:09.766594 1885 tablet_service.cc:1467] Processing CreateTablet for tablet 5743a4d1ce9445129f91730ac1f42159 (DEFAULT_TABLE table=test-table [id=ebdf3590dd664d4aa54d6d5943dd7d10]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:06:09.766765 2459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:09.759703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42266 (local address 127.11.254.131:45447)
0812 21:06:09.759874 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0812 21:06:09.759879 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 21:06:09.760413 (+ 534us) server_negotiation.cc:407] Connection header received
0812 21:06:09.760579 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:06:09.760582 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:06:09.760598 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:06:09.760651 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:06:09.761022 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:06:09.762231 (+ 1209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:06:09.765171 (+ 2940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:06:09.766231 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:06:09.766286 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:06:09.766360 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0812 21:06:09.766630 (+ 270us) server_negotiation.cc:299] Negotiation successful
0812 21:06:09.766683 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":77,"threads_started":1}
I20260812 21:06:09.767045 1885 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5743a4d1ce9445129f91730ac1f42159. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:06:09.770445 2291 tablet_service.cc:1467] Processing CreateTablet for tablet 5743a4d1ce9445129f91730ac1f42159 (DEFAULT_TABLE table=test-table [id=ebdf3590dd664d4aa54d6d5943dd7d10]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:06:09.770859 2291 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5743a4d1ce9445129f91730ac1f42159. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:06:09.770882 2470 tablet_bootstrap.cc:492] T 5743a4d1ce9445129f91730ac1f42159 P 5248b7f95db2494c8ef3d1bad13ac18b: Bootstrap starting.
I20260812 21:06:09.771821 2470 tablet_bootstrap.cc:654] T 5743a4d1ce9445129f91730ac1f42159 P 5248b7f95db2494c8ef3d1bad13ac18b: Neither blocks nor log segments found. Creating new log.
I20260812 21:06:09.772146 2470 log.cc:824] T 5743a4d1ce9445129f91730ac1f42159 P 5248b7f95db2494c8ef3d1bad13ac18b: Log is configured to *not* fsync() on all Append() calls
I20260812 21:06:09.774209 2470 tablet_bootstrap.cc:492] T 5743a4d1ce9445129f91730ac1f42159 P 5248b7f95db2494c8ef3d1bad13ac18b: No bootstrap required, opened a new log
I20260812 21:06:09.774442 2470 ts_tablet_manager.cc:1397] T 5743a4d1ce9445129f91730ac1f42159 P 5248b7f95db2494c8ef3d1bad13ac18b: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260812 21:06:09.777243 2470 raft_consensus.cc:348] T 5743a4d1ce9445129f91730ac1f42159 P 5248b7f95db2494c8ef3d1bad13ac18b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80cb37a528d743bb9fc081e1010b5fe3" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 45447 } } peers { permanent_uuid: "f7fe70f4546147589958c6297ea17bda" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 37341 } } peers { permanent_uuid: "5248b7f95db2494c8ef3d1bad13ac18b" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 33855 } }
I20260812 21:06:09.777575 2470 raft_consensus.cc:374] T 5743a4d1ce9445129f91730ac1f42159 P 5248b7f95db2494c8ef3d1bad13ac18b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:06:09.777675 2470 raft_consensus.cc:729] T 5743a4d1ce9445129f91730ac1f42159 P 5248b7f95db2494c8ef3d1bad13ac18b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5248b7f95db2494c8ef3d1bad13ac18b, State: Initialized, Role: FOLLOWER
I20260812 21:06:09.777911 2470 consensus_queue.cc:260] T 5743a4d1ce9445129f91730ac1f42159 P 5248b7f95db2494c8ef3d1bad13ac18b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80cb37a528d743bb9fc081e1010b5fe3" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 45447 } } peers { permanent_uuid: "f7fe70f4546147589958c6297ea17bda" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 37341 } } peers { permanent_uuid: "5248b7f95db2494c8ef3d1bad13ac18b" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 33855 } }
I20260812 21:06:09.778380 2470 ts_tablet_manager.cc:1428] T 5743a4d1ce9445129f91730ac1f42159 P 5248b7f95db2494c8ef3d1bad13ac18b: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260812 21:06:09.778561 1986 heartbeater.cc:503] Master 127.11.254.190:40171 was elected leader, sending a full tablet report...
I20260812 21:06:09.779906 2457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:09.760126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.130:37341 (local address 127.0.0.1:46278)
0812 21:06:09.760857 (+ 731us) negotiation.cc:107] Waiting for socket to connect
0812 21:06:09.760866 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 21:06:09.760921 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:06:09.775523 (+ 14602us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:06:09.775524 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:06:09.775528 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:06:09.775607 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:06:09.775609 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:06:09.777018 (+ 1409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:06:09.777019 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:06:09.778281 (+ 1262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:06:09.778285 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:06:09.779644 (+ 1359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:06:09.779645 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:06:09.779726 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:06:09.779732 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 21:06:09.779764 (+ 32us) client_negotiation.cc:240] Negotiation successful
0812 21:06:09.779815 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":713,"thread_start_us":85,"threads_started":1}
I20260812 21:06:09.780405 2465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:09.760235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46278 (local address 127.11.254.130:37341)
0812 21:06:09.775321 (+ 15086us) server_negotiation.cc:206] Beginning negotiation
0812 21:06:09.775326 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 21:06:09.775339 (+ 13us) server_negotiation.cc:407] Connection header received
0812 21:06:09.775377 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:06:09.775380 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:06:09.775397 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:06:09.775708 (+ 311us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:06:09.775804 (+ 96us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:06:09.776902 (+ 1098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:06:09.778737 (+ 1835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:06:09.779523 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:06:09.780161 (+ 638us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:06:09.780245 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0812 21:06:09.780273 (+ 28us) server_negotiation.cc:299] Negotiation successful
0812 21:06:09.780317 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15041,"thread_start_us":57,"threads_started":1}
I20260812 21:06:09.781286 2110 tablet_service.cc:1467] Processing CreateTablet for tablet 5743a4d1ce9445129f91730ac1f42159 (DEFAULT_TABLE table=test-table [id=ebdf3590dd664d4aa54d6d5943dd7d10]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:06:09.781708 2110 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5743a4d1ce9445129f91730ac1f42159. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:06:09.782140 2473 tablet_bootstrap.cc:492] T 5743a4d1ce9445129f91730ac1f42159 P 80cb37a528d743bb9fc081e1010b5fe3: Bootstrap starting.
I20260812 21:06:09.782888 2473 tablet_bootstrap.cc:654] T 5743a4d1ce9445129f91730ac1f42159 P 80cb37a528d743bb9fc081e1010b5fe3: Neither blocks nor log segments found. Creating new log.
I20260812 21:06:09.783252 2473 log.cc:824] T 5743a4d1ce9445129f91730ac1f42159 P 80cb37a528d743bb9fc081e1010b5fe3: Log is configured to *not* fsync() on all Append() calls
I20260812 21:06:09.785547 2473 tablet_bootstrap.cc:492] T 5743a4d1ce9445129f91730ac1f42159 P 80cb37a528d743bb9fc081e1010b5fe3: No bootstrap required, opened a new log
I20260812 21:06:09.785702 2473 ts_tablet_manager.cc:1397] T 5743a4d1ce9445129f91730ac1f42159 P 80cb37a528d743bb9fc081e1010b5fe3: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260812 21:06:09.790015 2473 raft_consensus.cc:348] T 5743a4d1ce9445129f91730ac1f42159 P 80cb37a528d743bb9fc081e1010b5fe3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80cb37a528d743bb9fc081e1010b5fe3" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 45447 } } peers { permanent_uuid: "f7fe70f4546147589958c6297ea17bda" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 37341 } } peers { permanent_uuid: "5248b7f95db2494c8ef3d1bad13ac18b" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 33855 } }
I20260812 21:06:09.790428 2473 raft_consensus.cc:374] T 5743a4d1ce9445129f91730ac1f42159 P 80cb37a528d743bb9fc081e1010b5fe3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:06:09.790660 2473 raft_consensus.cc:729] T 5743a4d1ce9445129f91730ac1f42159 P 80cb37a528d743bb9fc081e1010b5fe3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80cb37a528d743bb9fc081e1010b5fe3, State: Initialized, Role: FOLLOWER
I20260812 21:06:09.791069 2473 consensus_queue.cc:260] T 5743a4d1ce9445129f91730ac1f42159 P 80cb37a528d743bb9fc081e1010b5fe3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80cb37a528d743bb9fc081e1010b5fe3" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 45447 } } peers { permanent_uuid: "f7fe70f4546147589958c6297ea17bda" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 37341 } } peers { permanent_uuid: "5248b7f95db2494c8ef3d1bad13ac18b" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 33855 } }
I20260812 21:06:09.791469 2473 ts_tablet_manager.cc:1428] T 5743a4d1ce9445129f91730ac1f42159 P 80cb37a528d743bb9fc081e1010b5fe3: Time spent starting tablet: real 0.006s user 0.003s sys 0.000s
I20260812 21:06:09.791649 2415 heartbeater.cc:503] Master 127.11.254.190:40171 was elected leader, sending a full tablet report...
I20260812 21:06:09.792883 2479 tablet_bootstrap.cc:492] T 5743a4d1ce9445129f91730ac1f42159 P f7fe70f4546147589958c6297ea17bda: Bootstrap starting.
I20260812 21:06:09.795805 2479 tablet_bootstrap.cc:654] T 5743a4d1ce9445129f91730ac1f42159 P f7fe70f4546147589958c6297ea17bda: Neither blocks nor log segments found. Creating new log.
I20260812 21:06:09.797159 2479 log.cc:824] T 5743a4d1ce9445129f91730ac1f42159 P f7fe70f4546147589958c6297ea17bda: Log is configured to *not* fsync() on all Append() calls
I20260812 21:06:09.800297 2479 tablet_bootstrap.cc:492] T 5743a4d1ce9445129f91730ac1f42159 P f7fe70f4546147589958c6297ea17bda: No bootstrap required, opened a new log
I20260812 21:06:09.800431 2479 ts_tablet_manager.cc:1397] T 5743a4d1ce9445129f91730ac1f42159 P f7fe70f4546147589958c6297ea17bda: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20260812 21:06:09.803781 2479 raft_consensus.cc:348] T 5743a4d1ce9445129f91730ac1f42159 P f7fe70f4546147589958c6297ea17bda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80cb37a528d743bb9fc081e1010b5fe3" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 45447 } } peers { permanent_uuid: "f7fe70f4546147589958c6297ea17bda" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 37341 } } peers { permanent_uuid: "5248b7f95db2494c8ef3d1bad13ac18b" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 33855 } }
I20260812 21:06:09.804479 2479 raft_consensus.cc:374] T 5743a4d1ce9445129f91730ac1f42159 P f7fe70f4546147589958c6297ea17bda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:06:09.804579 2479 raft_consensus.cc:729] T 5743a4d1ce9445129f91730ac1f42159 P f7fe70f4546147589958c6297ea17bda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7fe70f4546147589958c6297ea17bda, State: Initialized, Role: FOLLOWER
I20260812 21:06:09.805325 2479 consensus_queue.cc:260] T 5743a4d1ce9445129f91730ac1f42159 P f7fe70f4546147589958c6297ea17bda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80cb37a528d743bb9fc081e1010b5fe3" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 45447 } } peers { permanent_uuid: "f7fe70f4546147589958c6297ea17bda" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 37341 } } peers { permanent_uuid: "5248b7f95db2494c8ef3d1bad13ac18b" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 33855 } }
I20260812 21:06:09.805759 2479 ts_tablet_manager.cc:1428] T 5743a4d1ce9445129f91730ac1f42159 P f7fe70f4546147589958c6297ea17bda: Time spent starting tablet: real 0.005s user 0.003s sys 0.000s
I20260812 21:06:09.805938 2216 heartbeater.cc:503] Master 127.11.254.190:40171 was elected leader, sending a full tablet report...
W20260812 21:06:09.860924 1987 tablet.cc:2369] T 5743a4d1ce9445129f91730ac1f42159 P 5248b7f95db2494c8ef3d1bad13ac18b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 21:06:09.965502 2416 tablet.cc:2369] T 5743a4d1ce9445129f91730ac1f42159 P 80cb37a528d743bb9fc081e1010b5fe3: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 12 21:06:10 dist-test-slave-vpvm krb5kdc[32590](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568769, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.190@KRBTEST.COM for HTTP/127.11.254.148@KRBTEST.COM
W20260812 21:06:10.043606 2217 tablet.cc:2369] T 5743a4d1ce9445129f91730ac1f42159 P f7fe70f4546147589958c6297ea17bda: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 21:06:10.075990 2474 raft_consensus.cc:482] T 5743a4d1ce9445129f91730ac1f42159 P 5248b7f95db2494c8ef3d1bad13ac18b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 21:06:10.076155 2474 raft_consensus.cc:504] T 5743a4d1ce9445129f91730ac1f42159 P 5248b7f95db2494c8ef3d1bad13ac18b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80cb37a528d743bb9fc081e1010b5fe3" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 45447 } } peers { permanent_uuid: "f7fe70f4546147589958c6297ea17bda" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 37341 } } peers { permanent_uuid: "5248b7f95db2494c8ef3d1bad13ac18b" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 33855 } }
I20260812 21:06:10.076601 2474 leader_election.cc:288] T 5743a4d1ce9445129f91730ac1f42159 P 5248b7f95db2494c8ef3d1bad13ac18b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80cb37a528d743bb9fc081e1010b5fe3 (127.11.254.131:45447), f7fe70f4546147589958c6297ea17bda (127.11.254.130:37341)
I20260812 21:06:10.082345 2465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:10.077209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.129:43859 (local address 127.11.254.130:37341)
0812 21:06:10.077259 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0812 21:06:10.077261 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:06:10.077589 (+ 328us) server_negotiation.cc:407] Connection header received
0812 21:06:10.077628 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:06:10.077629 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:06:10.077640 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:06:10.077695 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:06:10.077863 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:06:10.078649 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:06:10.080154 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:06:10.080746 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:06:10.080798 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:06:10.080808 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0812 21:06:10.082213 (+ 1405us) server_negotiation.cc:299] Negotiation successful
0812 21:06:10.082265 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 21:06:10.082849 2487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:10.077107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.130:37341 (local address 127.11.254.129:43859)
0812 21:06:10.077491 (+ 384us) negotiation.cc:107] Waiting for socket to connect
0812 21:06:10.077498 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0812 21:06:10.077542 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:06:10.077709 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:06:10.077710 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:06:10.077712 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:06:10.077793 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:06:10.077795 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:06:10.078755 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:06:10.078757 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:06:10.080030 (+ 1273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:06:10.080033 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:06:10.082025 (+ 1992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:06:10.082026 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:06:10.082118 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:06:10.082123 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 21:06:10.082692 (+ 569us) client_negotiation.cc:240] Negotiation successful
0812 21:06:10.082743 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":372,"thread_start_us":74,"threads_started":1}
I20260812 21:06:10.083415 2141 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5743a4d1ce9445129f91730ac1f42159" candidate_uuid: "5248b7f95db2494c8ef3d1bad13ac18b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7fe70f4546147589958c6297ea17bda" is_pre_election: true
I20260812 21:06:10.083597 2141 raft_consensus.cc:2454] T 5743a4d1ce9445129f91730ac1f42159 P f7fe70f4546147589958c6297ea17bda [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5248b7f95db2494c8ef3d1bad13ac18b in term 0.
I20260812 21:06:10.083876 1805 leader_election.cc:302] T 5743a4d1ce9445129f91730ac1f42159 P 5248b7f95db2494c8ef3d1bad13ac18b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5248b7f95db2494c8ef3d1bad13ac18b, f7fe70f4546147589958c6297ea17bda; no voters:
I20260812 21:06:10.084024 2474 raft_consensus.cc:2781] T 5743a4d1ce9445129f91730ac1f42159 P 5248b7f95db2494c8ef3d1bad13ac18b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 21:06:10.084084 2474 raft_consensus.cc:482] T 5743a4d1ce9445129f91730ac1f42159 P 5248b7f95db2494c8ef3d1bad13ac18b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 21:06:10.084108 2474 raft_consensus.cc:3037] T 5743a4d1ce9445129f91730ac1f42159 P 5248b7f95db2494c8ef3d1bad13ac18b [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:06:10.085045 2474 raft_consensus.cc:504] T 5743a4d1ce9445129f91730ac1f42159 P 5248b7f95db2494c8ef3d1bad13ac18b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80cb37a528d743bb9fc081e1010b5fe3" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 45447 } } peers { permanent_uuid: "f7fe70f4546147589958c6297ea17bda" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 37341 } } peers { permanent_uuid: "5248b7f95db2494c8ef3d1bad13ac18b" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 33855 } }
I20260812 21:06:10.085134 2459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:10.076959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.129:58469 (local address 127.11.254.131:45447)
0812 21:06:10.078040 (+ 1081us) server_negotiation.cc:206] Beginning negotiation
0812 21:06:10.078042 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:06:10.078051 (+ 9us) server_negotiation.cc:407] Connection header received
0812 21:06:10.078081 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:06:10.078082 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:06:10.078092 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:06:10.078144 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:06:10.080361 (+ 2217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:06:10.081053 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:06:10.084190 (+ 3137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:06:10.084740 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:06:10.084789 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:06:10.084798 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 21:06:10.085018 (+ 220us) server_negotiation.cc:299] Negotiation successful
0812 21:06:10.085065 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1049}
I20260812 21:06:10.085229 2474 leader_election.cc:288] T 5743a4d1ce9445129f91730ac1f42159 P 5248b7f95db2494c8ef3d1bad13ac18b [CANDIDATE]: Term 1 election: Requested vote from peers 80cb37a528d743bb9fc081e1010b5fe3 (127.11.254.131:45447), f7fe70f4546147589958c6297ea17bda (127.11.254.130:37341)
I20260812 21:06:10.085369 2486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:10.076838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.131:45447 (local address 127.11.254.129:58469)
0812 21:06:10.077336 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0812 21:06:10.077348 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 21:06:10.077405 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:06:10.078196 (+ 791us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:06:10.078197 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:06:10.078199 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:06:10.078269 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:06:10.078270 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:06:10.081162 (+ 2892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:06:10.081163 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:06:10.084082 (+ 2919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:06:10.084086 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:06:10.084857 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:06:10.084857 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:06:10.084941 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:06:10.084946 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 21:06:10.085246 (+ 300us) client_negotiation.cc:240] Negotiation successful
0812 21:06:10.085289 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":476,"thread_start_us":119,"threads_started":1}
I20260812 21:06:10.085774 2343 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5743a4d1ce9445129f91730ac1f42159" candidate_uuid: "5248b7f95db2494c8ef3d1bad13ac18b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80cb37a528d743bb9fc081e1010b5fe3"
I20260812 21:06:10.085798 2344 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5743a4d1ce9445129f91730ac1f42159" candidate_uuid: "5248b7f95db2494c8ef3d1bad13ac18b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80cb37a528d743bb9fc081e1010b5fe3" is_pre_election: true
I20260812 21:06:10.085914 2343 raft_consensus.cc:3037] T 5743a4d1ce9445129f91730ac1f42159 P 80cb37a528d743bb9fc081e1010b5fe3 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:06:10.086807 2343 raft_consensus.cc:2454] T 5743a4d1ce9445129f91730ac1f42159 P 80cb37a528d743bb9fc081e1010b5fe3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5248b7f95db2494c8ef3d1bad13ac18b in term 1.
I20260812 21:06:10.086990 2141 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5743a4d1ce9445129f91730ac1f42159" candidate_uuid: "5248b7f95db2494c8ef3d1bad13ac18b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7fe70f4546147589958c6297ea17bda"
I20260812 21:06:10.087015 1806 leader_election.cc:302] T 5743a4d1ce9445129f91730ac1f42159 P 5248b7f95db2494c8ef3d1bad13ac18b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5248b7f95db2494c8ef3d1bad13ac18b, 80cb37a528d743bb9fc081e1010b5fe3; no voters:
I20260812 21:06:10.087100 2474 raft_consensus.cc:2781] T 5743a4d1ce9445129f91730ac1f42159 P 5248b7f95db2494c8ef3d1bad13ac18b [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:06:10.087071 2141 raft_consensus.cc:3037] T 5743a4d1ce9445129f91730ac1f42159 P f7fe70f4546147589958c6297ea17bda [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:06:10.087255 2474 raft_consensus.cc:686] T 5743a4d1ce9445129f91730ac1f42159 P 5248b7f95db2494c8ef3d1bad13ac18b [term 1 LEADER]: Becoming Leader. State: Replica: 5248b7f95db2494c8ef3d1bad13ac18b, State: Running, Role: LEADER
I20260812 21:06:10.087352 2474 consensus_queue.cc:237] T 5743a4d1ce9445129f91730ac1f42159 P 5248b7f95db2494c8ef3d1bad13ac18b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80cb37a528d743bb9fc081e1010b5fe3" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 45447 } } peers { permanent_uuid: "f7fe70f4546147589958c6297ea17bda" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 37341 } } peers { permanent_uuid: "5248b7f95db2494c8ef3d1bad13ac18b" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 33855 } }
I20260812 21:06:10.087855 2141 raft_consensus.cc:2454] T 5743a4d1ce9445129f91730ac1f42159 P f7fe70f4546147589958c6297ea17bda [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5248b7f95db2494c8ef3d1bad13ac18b in term 1.
I20260812 21:06:10.088161 1669 catalog_manager.cc:5495] T 5743a4d1ce9445129f91730ac1f42159 P 5248b7f95db2494c8ef3d1bad13ac18b reported cstate change: term changed from 0 to 1, leader changed from <none> to 5248b7f95db2494c8ef3d1bad13ac18b (127.11.254.129). New cstate: current_term: 1 leader_uuid: "5248b7f95db2494c8ef3d1bad13ac18b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80cb37a528d743bb9fc081e1010b5fe3" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 45447 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f7fe70f4546147589958c6297ea17bda" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 37341 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5248b7f95db2494c8ef3d1bad13ac18b" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 33855 } health_report { overall_health: HEALTHY } } }
I20260812 21:06:10.133857 2460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:10.130272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48962 (local address 127.11.254.129:33855)
0812 21:06:10.130336 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0812 21:06:10.130338 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:06:10.130350 (+ 12us) server_negotiation.cc:407] Connection header received
0812 21:06:10.130468 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:06:10.130470 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:06:10.130496 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:06:10.130581 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 21:06:10.130827 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:06:10.131592 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:06:10.132710 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:06:10.133013 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:06:10.133078 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:06:10.133319 (+ 241us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 21:06:10.133464 (+ 145us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 21:06:10.133548 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0812 21:06:10.133747 (+ 199us) server_negotiation.cc:299] Negotiation successful
0812 21:06:10.133774 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260812 21:06:10.151289 2141 raft_consensus.cc:1261] T 5743a4d1ce9445129f91730ac1f42159 P f7fe70f4546147589958c6297ea17bda [term 1 FOLLOWER]: Refusing update from remote peer 5248b7f95db2494c8ef3d1bad13ac18b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 21:06:10.151773 2343 raft_consensus.cc:1261] T 5743a4d1ce9445129f91730ac1f42159 P 80cb37a528d743bb9fc081e1010b5fe3 [term 1 FOLLOWER]: Refusing update from remote peer 5248b7f95db2494c8ef3d1bad13ac18b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 21:06:10.152240 2474 consensus_queue.cc:1035] T 5743a4d1ce9445129f91730ac1f42159 P 5248b7f95db2494c8ef3d1bad13ac18b [LEADER]: Connected to new peer: Peer: permanent_uuid: "80cb37a528d743bb9fc081e1010b5fe3" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 45447 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 21:06:10.152747 2474 consensus_queue.cc:1035] T 5743a4d1ce9445129f91730ac1f42159 P 5248b7f95db2494c8ef3d1bad13ac18b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f7fe70f4546147589958c6297ea17bda" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 37341 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 21:06:10.163635 2495 mvcc.cc:205] Tried to move back new op lower bound from 7317785682535936000 to 7317785682278354944. Current Snapshot: MvccSnapshot[applied={T|T < 7317785682535936000}]
I20260812 21:06:10.166543 2500 mvcc.cc:205] Tried to move back new op lower bound from 7317785682535936000 to 7317785682278354944. Current Snapshot: MvccSnapshot[applied={T|T < 7317785682535936000}]
I20260812 21:06:10.173245 1669 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60930:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 21:06:10.173378 1669 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60930:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 21:06:10.174480 1669 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ca4c639e0fef454a830a71a98575fb7a: Sending DeleteTablet for 3 replicas of tablet 5743a4d1ce9445129f91730ac1f42159
I20260812 21:06:10.175002 2291 tablet_service.cc:1514] Processing DeleteTablet for tablet 5743a4d1ce9445129f91730ac1f42159 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:06:10 UTC) from {username='kudu', principal='kudu/127.11.254.190@KRBTEST.COM'} at 127.0.0.1:42266
I20260812 21:06:10.175051 1885 tablet_service.cc:1514] Processing DeleteTablet for tablet 5743a4d1ce9445129f91730ac1f42159 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:06:10 UTC) from {username='kudu', principal='kudu/127.11.254.190@KRBTEST.COM'} at 127.0.0.1:48956
I20260812 21:06:10.175551 2110 tablet_service.cc:1514] Processing DeleteTablet for tablet 5743a4d1ce9445129f91730ac1f42159 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:06:10 UTC) from {username='kudu', principal='kudu/127.11.254.190@KRBTEST.COM'} at 127.0.0.1:46278
I20260812 21:06:10.176355 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 1736
I20260812 21:06:10.186658 2515 tablet_replica.cc:330] T 5743a4d1ce9445129f91730ac1f42159 P f7fe70f4546147589958c6297ea17bda: stopping tablet replica
I20260812 21:06:10.186822 2515 raft_consensus.cc:2229] T 5743a4d1ce9445129f91730ac1f42159 P f7fe70f4546147589958c6297ea17bda [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 21:06:10.186933 2515 raft_consensus.cc:2258] T 5743a4d1ce9445129f91730ac1f42159 P f7fe70f4546147589958c6297ea17bda [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 21:06:10.189167 2515 ts_tablet_manager.cc:1905] T 5743a4d1ce9445129f91730ac1f42159 P f7fe70f4546147589958c6297ea17bda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 21:06:10.190081 2511 tablet_replica.cc:330] T 5743a4d1ce9445129f91730ac1f42159 P 80cb37a528d743bb9fc081e1010b5fe3: stopping tablet replica
I20260812 21:06:10.190231 2511 raft_consensus.cc:2229] T 5743a4d1ce9445129f91730ac1f42159 P 80cb37a528d743bb9fc081e1010b5fe3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 21:06:10.191185 2511 raft_consensus.cc:2258] T 5743a4d1ce9445129f91730ac1f42159 P 80cb37a528d743bb9fc081e1010b5fe3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 21:06:10.191533 2515 ts_tablet_manager.cc:1918] T 5743a4d1ce9445129f91730ac1f42159 P f7fe70f4546147589958c6297ea17bda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 21:06:10.191653 2515 log.cc:1196] T 5743a4d1ce9445129f91730ac1f42159 P f7fe70f4546147589958c6297ea17bda: Deleting WAL directory at /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/minicluster-data/ts-1/wal/wals/5743a4d1ce9445129f91730ac1f42159
I20260812 21:06:10.191991 2515 ts_tablet_manager.cc:1939] T 5743a4d1ce9445129f91730ac1f42159 P f7fe70f4546147589958c6297ea17bda: Deleting consensus metadata
I20260812 21:06:10.192966 1641 catalog_manager.cc:4841] TS f7fe70f4546147589958c6297ea17bda (127.11.254.130:37341): tablet 5743a4d1ce9445129f91730ac1f42159 (table test-table [id=ebdf3590dd664d4aa54d6d5943dd7d10]) successfully deleted
I20260812 21:06:10.193547 2511 ts_tablet_manager.cc:1905] T 5743a4d1ce9445129f91730ac1f42159 P 80cb37a528d743bb9fc081e1010b5fe3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 21:06:10.195263 2511 ts_tablet_manager.cc:1918] T 5743a4d1ce9445129f91730ac1f42159 P 80cb37a528d743bb9fc081e1010b5fe3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 21:06:10.195397 2511 log.cc:1196] T 5743a4d1ce9445129f91730ac1f42159 P 80cb37a528d743bb9fc081e1010b5fe3: Deleting WAL directory at /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786568607354921-12282-0/minicluster-data/ts-2/wal/wals/5743a4d1ce9445129f91730ac1f42159
I20260812 21:06:10.195744 2511 ts_tablet_manager.cc:1939] T 5743a4d1ce9445129f91730ac1f42159 P 80cb37a528d743bb9fc081e1010b5fe3: Deleting consensus metadata
I20260812 21:06:10.196377 1643 catalog_manager.cc:4841] TS 80cb37a528d743bb9fc081e1010b5fe3 (127.11.254.131:45447): tablet 5743a4d1ce9445129f91730ac1f42159 (table test-table [id=ebdf3590dd664d4aa54d6d5943dd7d10]) successfully deleted
W20260812 21:06:10.201745 1645 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.254.129:33855 (error 108)
I20260812 21:06:10.202659 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 1989
I20260812 21:06:10.211331 2457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:10.211156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.129:33855 (local address 127.0.0.1:48968)
0812 21:06:10.211219 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0812 21:06:10.211280 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.129:33855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260812 21:06:10.211686 1645 catalog_manager.cc:4584] TS 5248b7f95db2494c8ef3d1bad13ac18b (127.11.254.129:33855): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5743a4d1ce9445129f91730ac1f42159: Network error: Client connection negotiation failed: client connection to 127.11.254.129:33855: connect: Connection refused (error 111)
I20260812 21:06:10.215898 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 2219
I20260812 21:06:10.264353 2457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:10.264195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.129:33855 (local address 127.0.0.1:48978)
0812 21:06:10.264263 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0812 21:06:10.264301 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.129:33855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260812 21:06:10.264963 2457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:10.264876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.129:33855 (local address 127.0.0.1:48990)
0812 21:06:10.264916 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0812 21:06:10.264936 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.129:33855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260812 21:06:10.265389 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 1620
I20260812 21:06:10.286955 12282 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260812 21:06:10.631845 12282 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260812 21:06:10.631961 12282 mini_ranger.cc:66] Stopping Ranger...
I20260812 21:06:10.976853 12282 mini_ranger.cc:68] Stopped Ranger
2026-08-12 21:06:10.977 UTC [32634] LOG: received smart shutdown request
2026-08-12 21:06:10.981 UTC [32634] LOG: background worker "logical replication launcher" (PID 32646) exited with exit code 1
2026-08-12 21:06:10.982 UTC [32641] LOG: shutting down
2026-08-12 21:06:11.152 UTC [32634] LOG: database system is shut down
2026-08-12T21:06:11Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (50561 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786568607354921-12282-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 21:06:11 dist-test-slave-vpvm krb5kdc[2535](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 21:06:11 dist-test-slave-vpvm krb5kdc[2535](info): set up 2 sockets
Aug 12 21:06:11 dist-test-slave-vpvm krb5kdc[2535](info): commencing operation
krb5kdc: starting...
W20260812 21:06:12.468233 12282 mini_kdc.cc:121] Time spent starting KDC: real 1.204s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786568607354921-12282-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786568607354921-12282-0/krb5kdc/test-user.keytab.
Aug 12 21:06:12 dist-test-slave-vpvm krb5kdc[2535](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568772, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T21:06:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T21:06:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.254.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.254.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.254.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.190.keytab.
Entry for principal kudu/127.11.254.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.190.keytab.
I20260812 21:06:12.594496 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786568607354921-12282-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786568607354921-12282-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786568607354921-12282-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.254.190:43331
--webserver_interface=127.11.254.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.190.keytab
--principal=kudu/127.11.254.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.254.148:42579
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.254.190:43331,127.11.254.189:36381,127.11.254.188:44539
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786568607354921-12282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786568607354921-12282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786568607354921-12282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:06:12.679080 2558 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:06:12.679296 2558 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:06:12.679318 2558 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:06:12.680508 2558 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 21:06:12.680555 2558 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:06:12.680567 2558 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:06:12.680579 2558 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 21:06:12.680590 2558 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 21:06:12.682140 2558 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.254.148:42579
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786568607354921-12282-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786568607354921-12282-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.254.190:43331,127.11.254.189:36381,127.11.254.188:44539
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.190.keytab
--principal=kudu/127.11.254.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.254.190:43331
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.254.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2558
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786568607354921-12282-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:06:12.682368 2558 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:06:12.682590 2558 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:06:12.687390 2563 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:06:12.687390 2564 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:06:12.687410 2566 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:06:12.687487 2558 server_base.cc:1034] running on GCE node
I20260812 21:06:12.687903 2558 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:06:12.688231 2558 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:06:12.689373 2558 hybrid_clock.cc:648] HybridClock initialized: now 1786568772689354 us; error 36 us; skew 500 ppm
Aug 12 21:06:12 dist-test-slave-vpvm krb5kdc[2535](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568772, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:06:12.691978 2558 init.cc:377] Logged in from keytab as kudu/127.11.254.190@KRBTEST.COM (short username kudu)
I20260812 21:06:12.692430 2558 webserver.cc:460] Webserver started at http://127.11.254.190:41731/ using document root <none> and password file <none>
I20260812 21:06:12.692639 2558 fs_manager.cc:359] Metadata directory not provided
I20260812 21:06:12.692687 2558 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:06:12.692814 2558 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:06:12.693908 2558 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786568607354921-12282-0/minicluster-data/master-0/data/instance:
uuid: "eeab60837d1942369667c7c668515091"
format_stamp: "Formatted at 2026-08-12 21:06:12 on dist-test-slave-vpvm"
server_key: "6a9bd4b66d3f698018f63b75cbe2b680"
server_key_iv: "a8ba76615d59a809d8ebda1969cb21ad"
server_key_version: "encryptionkey@0"
I20260812 21:06:12.694311 2558 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786568607354921-12282-0/minicluster-data/master-0/wal/instance:
uuid: "eeab60837d1942369667c7c668515091"
format_stamp: "Formatted at 2026-08-12 21:06:12 on dist-test-slave-vpvm"
server_key: "6a9bd4b66d3f698018f63b75cbe2b680"
server_key_iv: "a8ba76615d59a809d8ebda1969cb21ad"
server_key_version: "encryptionkey@0"
I20260812 21:06:12.695683 2558 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 21:06:12.696431 2584 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:06:12.696583 2558 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 21:06:12.696699 2558 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786568607354921-12282-0/minicluster-data/master-0/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786568607354921-12282-0/minicluster-data/master-0/wal
uuid: "eeab60837d1942369667c7c668515091"
format_stamp: "Formatted at 2026-08-12 21:06:12 on dist-test-slave-vpvm"
server_key: "6a9bd4b66d3f698018f63b75cbe2b680"
server_key_iv: "a8ba76615d59a809d8ebda1969cb21ad"
server_key_version: "encryptionkey@0"
I20260812 21:06:12.696770 2558 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786568607354921-12282-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786568607354921-12282-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786568607354921-12282-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:06:12.703873 2558 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:06:12.705991 2558 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:06:12.706151 2558 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:06:12.710208 2558 rpc_server.cc:307] RPC server started. Bound to: 127.11.254.190:43331
I20260812 21:06:12.710243 2660 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.254.190:43331 every 8 connection(s)
I20260812 21:06:12.710593 2558 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
I20260812 21:06:12.711607 2663 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:06:12.713238 2663 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.254.190" port: 43331 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 21:06:12 dist-test-slave-vpvm krb5kdc[2535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568772, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.190@KRBTEST.COM for kudu/127.11.254.190@KRBTEST.COM
I20260812 21:06:12.720290 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 2558
I20260812 21:06:12.720397 12282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786568607354921-12282-0/minicluster-data/master-0/wal/instance
I20260812 21:06:12.720608 12282 external_mini_cluster.cc:1428] Setting key 40b1fe9c471543aa32dc115fe1c89caa
I20260812 21:06:12.723376 2665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:12.713995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:43331 (local address 127.0.0.1:37308)
0812 21:06:12.714182 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0812 21:06:12.714224 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 21:06:12.714704 (+ 480us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:06:12.714946 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:06:12.714948 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:06:12.715206 (+ 258us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:06:12.715379 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:06:12.715384 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:06:12.716432 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:06:12.716433 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:06:12.717206 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:06:12.717209 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:06:12.717793 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:06:12.717794 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:06:12.717865 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:06:12.718429 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:06:12.718438 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:06:12.719904 (+ 1466us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:06:12.721706 (+ 1802us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:06:12.721715 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:06:12.721717 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:06:12.722002 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:06:12.722245 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:06:12.722247 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:06:12.722247 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:06:12.722368 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:06:12.722868 (+ 500us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:06:12.722875 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:06:12.722942 (+ 67us) client_negotiation.cc:769] Sending connection context
0812 21:06:12.723024 (+ 82us) client_negotiation.cc:240] Negotiation successful
0812 21:06:12.723192 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":81,"threads_started":1}
I20260812 21:06:12.723384 2666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:12.714034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37308 (local address 127.11.254.190:43331)
0812 21:06:12.714204 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0812 21:06:12.714215 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 21:06:12.714326 (+ 111us) server_negotiation.cc:407] Connection header received
0812 21:06:12.714827 (+ 501us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:06:12.714829 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:06:12.714847 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:06:12.714933 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:06:12.715481 (+ 548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:06:12.716338 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:06:12.717309 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:06:12.717686 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:06:12.717779 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:06:12.719988 (+ 2209us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:06:12.719995 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:06:12.719996 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:06:12.720003 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:06:12.721606 (+ 1603us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:06:12.722078 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:06:12.722081 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:06:12.722081 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:06:12.722174 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:06:12.722435 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:06:12.722436 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:06:12.722437 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:06:12.722792 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:06:12.722877 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0812 21:06:12.723075 (+ 198us) server_negotiation.cc:299] Negotiation successful
0812 21:06:12.723282 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":86,"server-negotiator.queue_time_us":133,"thread_start_us":84,"threads_started":1}
I20260812 21:06:12.724179 2663 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.254.189" port: 36381 } has no permanent_uuid. Determining permanent_uuid...
I20260812 21:06:12.724542 2665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:12.724429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.189:36381 (local address 127.0.0.1:32904)
0812 21:06:12.724473 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0812 21:06:12.724519 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.189:36381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260812 21:06:12.724694 2587 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.254.189:36381: connect: Connection refused (error 111)
I20260812 21:06:12.725162 2665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:12.725055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.189:36381 (local address 127.0.0.1:32910)
0812 21:06:12.725098 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0812 21:06:12.725143 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.189:36381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260812 21:06:12.725273 2663 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.254.189:36381: Network error: Client connection negotiation failed: client connection to 127.11.254.189:36381: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.11.254.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.254.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.254.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.189.keytab.
Entry for principal kudu/127.11.254.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.189.keytab.
I20260812 21:06:12.749341 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786568607354921-12282-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786568607354921-12282-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786568607354921-12282-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786568607354921-12282-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.254.189:36381
--webserver_interface=127.11.254.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.189.keytab
--principal=kudu/127.11.254.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.254.148:42579
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.254.190:43331,127.11.254.189:36381,127.11.254.188:44539
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786568607354921-12282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786568607354921-12282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786568607354921-12282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 21:06:12.774523 2663 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.254.189" port: 36381 } attempt: 1
I20260812 21:06:12.775107 2665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:12.774942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.189:36381 (local address 127.0.0.1:32916)
0812 21:06:12.775011 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0812 21:06:12.775069 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.189:36381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260812 21:06:12.775452 2665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:12.775367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.189:36381 (local address 127.0.0.1:32930)
0812 21:06:12.775407 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0812 21:06:12.775433 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.189:36381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260812 21:06:12.775542 2663 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.254.189:36381: Network error: Client connection negotiation failed: client connection to 127.11.254.189:36381: connect: Connection refused (error 111)
W20260812 21:06:12.835114 2676 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:06:12.835335 2676 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:06:12.835358 2676 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:06:12.836534 2676 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 21:06:12.836570 2676 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:06:12.836582 2676 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:06:12.836593 2676 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 21:06:12.836604 2676 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 21:06:12.838172 2676 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.254.148:42579
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786568607354921-12282-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786568607354921-12282-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.11.254.190:43331,127.11.254.189:36381,127.11.254.188:44539
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.189.keytab
--principal=kudu/127.11.254.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.254.189:36381
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786568607354921-12282-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.11.254.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2676
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786568607354921-12282-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:06:12.838382 2676 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:06:12.838581 2676 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:06:12.843613 2690 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:06:12.843722 2676 server_base.cc:1034] running on GCE node
W20260812 21:06:12.843586 2687 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:06:12.843683 2663 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.254.189" port: 36381 } attempt: 2
W20260812 21:06:12.843632 2688 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:06:12.844142 2665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:12.843978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.189:36381 (local address 127.0.0.1:32932)
0812 21:06:12.844046 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0812 21:06:12.844080 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.189:36381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260812 21:06:12.844342 2676 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:06:12.844494 2665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:12.844402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.189:36381 (local address 127.0.0.1:32938)
0812 21:06:12.844443 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0812 21:06:12.844471 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.189:36381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260812 21:06:12.844570 2663 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.254.189:36381: Network error: Client connection negotiation failed: client connection to 127.11.254.189:36381: connect: Connection refused (error 111)
I20260812 21:06:12.844702 2676 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:06:12.845851 2676 hybrid_clock.cc:648] HybridClock initialized: now 1786568772845831 us; error 27 us; skew 500 ppm
Aug 12 21:06:12 dist-test-slave-vpvm krb5kdc[2535](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568772, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:06:12.848444 2676 init.cc:377] Logged in from keytab as kudu/127.11.254.189@KRBTEST.COM (short username kudu)
I20260812 21:06:12.848930 2676 webserver.cc:460] Webserver started at http://127.11.254.189:37791/ using document root <none> and password file <none>
I20260812 21:06:12.849162 2676 fs_manager.cc:359] Metadata directory not provided
I20260812 21:06:12.849212 2676 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:06:12.849346 2676 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:06:12.850507 2676 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786568607354921-12282-0/minicluster-data/master-1/data/instance:
uuid: "1c1144d3d59c4213b01603eef88e9932"
format_stamp: "Formatted at 2026-08-12 21:06:12 on dist-test-slave-vpvm"
server_key: "0701c646cc18d680db03eebab1f4bc64"
server_key_iv: "648fbe106dc9dcc2a608a908cac7ffdf"
server_key_version: "encryptionkey@0"
I20260812 21:06:12.850863 2676 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786568607354921-12282-0/minicluster-data/master-1/wal/instance:
uuid: "1c1144d3d59c4213b01603eef88e9932"
format_stamp: "Formatted at 2026-08-12 21:06:12 on dist-test-slave-vpvm"
server_key: "0701c646cc18d680db03eebab1f4bc64"
server_key_iv: "648fbe106dc9dcc2a608a908cac7ffdf"
server_key_version: "encryptionkey@0"
I20260812 21:06:12.852208 2676 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 21:06:12.852975 2706 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:06:12.853147 2676 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 21:06:12.853236 2676 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786568607354921-12282-0/minicluster-data/master-1/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786568607354921-12282-0/minicluster-data/master-1/wal
uuid: "1c1144d3d59c4213b01603eef88e9932"
format_stamp: "Formatted at 2026-08-12 21:06:12 on dist-test-slave-vpvm"
server_key: "0701c646cc18d680db03eebab1f4bc64"
server_key_iv: "648fbe106dc9dcc2a608a908cac7ffdf"
server_key_version: "encryptionkey@0"
I20260812 21:06:12.853305 2676 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786568607354921-12282-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786568607354921-12282-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786568607354921-12282-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:06:12.860539 2676 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:06:12.862701 2676 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:06:12.862838 2676 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:06:12.866974 2676 rpc_server.cc:307] RPC server started. Bound to: 127.11.254.189:36381
I20260812 21:06:12.867065 2777 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.254.189:36381 every 8 connection(s)
I20260812 21:06:12.867370 2676 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786568607354921-12282-0/minicluster-data/master-1/data/info.pb
I20260812 21:06:12.868242 2778 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:06:12.869897 2778 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.254.190" port: 43331 } has no permanent_uuid. Determining permanent_uuid...
I20260812 21:06:12.875389 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 2676
I20260812 21:06:12.875522 12282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786568607354921-12282-0/minicluster-data/master-1/wal/instance
I20260812 21:06:12.875736 12282 external_mini_cluster.cc:1428] Setting key 2d2bec6ce632fcaaf129c4909bde964e
Aug 12 21:06:12 dist-test-slave-vpvm krb5kdc[2535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568772, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.189@KRBTEST.COM for kudu/127.11.254.190@KRBTEST.COM
I20260812 21:06:12.878908 2666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:12.870655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37318 (local address 127.11.254.190:43331)
0812 21:06:12.870738 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0812 21:06:12.870741 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 21:06:12.870912 (+ 171us) server_negotiation.cc:407] Connection header received
0812 21:06:12.871381 (+ 469us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:06:12.871382 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:06:12.871394 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:06:12.871444 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:06:12.872158 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:06:12.872712 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:06:12.873749 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:06:12.873981 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:06:12.874045 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:06:12.876238 (+ 2193us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:06:12.876240 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:06:12.876240 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:06:12.876248 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:06:12.877536 (+ 1288us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:06:12.877940 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:06:12.877942 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:06:12.877942 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:06:12.878029 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:06:12.878264 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:06:12.878265 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:06:12.878265 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:06:12.878386 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:06:12.878442 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0812 21:06:12.878725 (+ 283us) server_negotiation.cc:299] Negotiation successful
0812 21:06:12.878816 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 21:06:12.879086 2782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:12.870625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:43331 (local address 127.0.0.1:37318)
0812 21:06:12.870795 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0812 21:06:12.870832 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0812 21:06:12.871300 (+ 468us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:06:12.871515 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:06:12.871518 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:06:12.871766 (+ 248us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:06:12.871986 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:06:12.872011 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:06:12.872820 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:06:12.872821 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:06:12.873678 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:06:12.873681 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:06:12.874050 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:06:12.874051 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:06:12.874184 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:06:12.874649 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:06:12.874657 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:06:12.876138 (+ 1481us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:06:12.877624 (+ 1486us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:06:12.877632 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:06:12.877634 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:06:12.877869 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:06:12.878090 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:06:12.878091 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:06:12.878092 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:06:12.878206 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:06:12.878443 (+ 237us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:06:12.878451 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:06:12.878604 (+ 153us) client_negotiation.cc:769] Sending connection context
0812 21:06:12.878694 (+ 90us) client_negotiation.cc:240] Negotiation successful
0812 21:06:12.878862 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":88,"threads_started":1}
I20260812 21:06:12.879653 2778 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.254.189" port: 36381 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.11.254.188@KRBTEST.COM; defaulting to no policy
Aug 12 21:06:12 dist-test-slave-vpvm krb5kdc[2535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568772, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.189@KRBTEST.COM for kudu/127.11.254.189@KRBTEST.COM
I20260812 21:06:12.886806 2782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:12.879932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.189:36381 (local address 127.0.0.1:32954)
0812 21:06:12.879984 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0812 21:06:12.879993 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 21:06:12.880030 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:06:12.880304 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:06:12.880304 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:06:12.880465 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:06:12.880529 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:06:12.880531 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:06:12.881389 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:06:12.881390 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:06:12.882177 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:06:12.882179 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:06:12.882647 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:06:12.882648 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:06:12.882730 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:06:12.883077 (+ 347us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:06:12.883082 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:06:12.884328 (+ 1246us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:06:12.885782 (+ 1454us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:06:12.885782 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:06:12.885783 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:06:12.885932 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:06:12.886121 (+ 189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:06:12.886122 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:06:12.886122 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:06:12.886176 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:06:12.886476 (+ 300us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:06:12.886477 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:06:12.886520 (+ 43us) client_negotiation.cc:769] Sending connection context
0812 21:06:12.886581 (+ 61us) client_negotiation.cc:240] Negotiation successful
0812 21:06:12.886683 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260812 21:06:12.886889 2786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:12.879979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32954 (local address 127.11.254.189:36381)
0812 21:06:12.880157 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0812 21:06:12.880162 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 21:06:12.880181 (+ 19us) server_negotiation.cc:407] Connection header received
0812 21:06:12.880200 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:06:12.880202 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:06:12.880213 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:06:12.880272 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:06:12.880611 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:06:12.881294 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:06:12.882251 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:06:12.882558 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:06:12.882628 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:06:12.884401 (+ 1773us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:06:12.884409 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:06:12.884409 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:06:12.884416 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:06:12.885707 (+ 1291us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:06:12.886008 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:06:12.886012 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:06:12.886012 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:06:12.886065 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:06:12.886224 (+ 159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:06:12.886225 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:06:12.886225 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:06:12.886395 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:06:12.886474 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0812 21:06:12.886623 (+ 149us) server_negotiation.cc:299] Negotiation successful
0812 21:06:12.886753 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":47,"server-negotiator.queue_time_us":149,"thread_start_us":107,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.254.188@KRBTEST.COM" created.
I20260812 21:06:12.887640 2778 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.254.188" port: 44539 } has no permanent_uuid. Determining permanent_uuid...
I20260812 21:06:12.887959 2782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:12.887852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.188:44539 (local address 127.0.0.1:44094)
0812 21:06:12.887892 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0812 21:06:12.887938 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.188:44539: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260812 21:06:12.888062 2710 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.254.188:44539: connect: Connection refused (error 111)
I20260812 21:06:12.888484 2782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:12.888392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.188:44539 (local address 127.0.0.1:44104)
0812 21:06:12.888435 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0812 21:06:12.888460 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.188:44539: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260812 21:06:12.888581 2778 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.254.188:44539: Network error: Client connection negotiation failed: client connection to 127.11.254.188:44539: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.254.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.188.keytab.
Entry for principal kudu/127.11.254.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.188.keytab.
I20260812 21:06:12.902501 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786568607354921-12282-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786568607354921-12282-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786568607354921-12282-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786568607354921-12282-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.254.188:44539
--webserver_interface=127.11.254.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.188.keytab
--principal=kudu/127.11.254.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.254.148:42579
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.254.190:43331,127.11.254.189:36381,127.11.254.188:44539
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786568607354921-12282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786568607354921-12282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786568607354921-12282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 21:06:12.935719 2663 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.254.189" port: 36381 } attempt: 3
I20260812 21:06:12.937811 2778 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.254.188" port: 44539 } attempt: 1
I20260812 21:06:12.938261 2782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:12.938118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.188:44539 (local address 127.0.0.1:44106)
0812 21:06:12.938179 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0812 21:06:12.938222 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.188:44539: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260812 21:06:12.938685 2782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:12.938590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.188:44539 (local address 127.0.0.1:44118)
0812 21:06:12.938631 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0812 21:06:12.938661 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.188:44539: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260812 21:06:12.938814 2778 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.254.188:44539: Network error: Client connection negotiation failed: client connection to 127.11.254.188:44539: connect: Connection refused (error 111)
Aug 12 21:06:12 dist-test-slave-vpvm krb5kdc[2535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568772, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.190@KRBTEST.COM for kudu/127.11.254.189@KRBTEST.COM
I20260812 21:06:12.943115 2786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:12.936261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32960 (local address 127.11.254.189:36381)
0812 21:06:12.936336 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0812 21:06:12.936339 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 21:06:12.936352 (+ 13us) server_negotiation.cc:407] Connection header received
0812 21:06:12.936386 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:06:12.936387 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:06:12.936401 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:06:12.936464 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:06:12.936806 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:06:12.937610 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:06:12.938457 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:06:12.938737 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:06:12.938835 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:06:12.940821 (+ 1986us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:06:12.940823 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:06:12.940823 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:06:12.940830 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:06:12.942153 (+ 1323us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:06:12.942498 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:06:12.942498 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:06:12.942499 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:06:12.942545 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:06:12.942708 (+ 163us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:06:12.942709 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:06:12.942709 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:06:12.942808 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:06:12.942850 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0812 21:06:12.942986 (+ 136us) server_negotiation.cc:299] Negotiation successful
0812 21:06:12.943045 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260812 21:06:12.943181 2665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:12.936194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.189:36381 (local address 127.0.0.1:32960)
0812 21:06:12.936263 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0812 21:06:12.936276 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 21:06:12.936340 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:06:12.936479 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:06:12.936480 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:06:12.936663 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:06:12.936736 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:06:12.936738 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:06:12.937690 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:06:12.937691 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:06:12.938372 (+ 681us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:06:12.938377 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:06:12.938843 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:06:12.938844 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:06:12.938940 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:06:12.939336 (+ 396us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:06:12.939342 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:06:12.940733 (+ 1391us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:06:12.942223 (+ 1490us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:06:12.942224 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:06:12.942224 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:06:12.942431 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:06:12.942593 (+ 162us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:06:12.942594 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:06:12.942594 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:06:12.942657 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:06:12.942853 (+ 196us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:06:12.942854 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:06:12.942904 (+ 50us) client_negotiation.cc:769] Sending connection context
0812 21:06:12.942974 (+ 70us) client_negotiation.cc:240] Negotiation successful
0812 21:06:12.943073 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260812 21:06:12.943593 2663 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.254.188" port: 44539 } has no permanent_uuid. Determining permanent_uuid...
I20260812 21:06:12.943949 2665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:12.943848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.188:44539 (local address 127.0.0.1:44134)
0812 21:06:12.943890 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0812 21:06:12.943924 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.188:44539: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260812 21:06:12.944242 2665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:12.944171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.188:44539 (local address 127.0.0.1:44144)
0812 21:06:12.944204 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0812 21:06:12.944227 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.188:44539: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260812 21:06:12.944324 2663 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.254.188:44539: Network error: Client connection negotiation failed: client connection to 127.11.254.188:44539: connect: Connection refused (error 111)
I20260812 21:06:12.975512 2663 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.254.188" port: 44539 } attempt: 1
I20260812 21:06:12.976060 2665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:12.975923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.188:44539 (local address 127.0.0.1:44146)
0812 21:06:12.975984 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0812 21:06:12.976029 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.188:44539: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260812 21:06:12.976408 2665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:12.976319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.188:44539 (local address 127.0.0.1:44158)
0812 21:06:12.976360 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0812 21:06:12.976385 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.188:44539: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260812 21:06:12.976501 2663 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.254.188:44539: Network error: Client connection negotiation failed: client connection to 127.11.254.188:44539: connect: Connection refused (error 111)
W20260812 21:06:12.988147 2790 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:06:12.988368 2790 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:06:12.988389 2790 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:06:12.989509 2790 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 21:06:12.989544 2790 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:06:12.989558 2790 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:06:12.989569 2790 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 21:06:12.989580 2790 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 21:06:12.991117 2790 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.254.148:42579
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786568607354921-12282-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786568607354921-12282-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.11.254.190:43331,127.11.254.189:36381,127.11.254.188:44539
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.188.keytab
--principal=kudu/127.11.254.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.254.188:44539
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786568607354921-12282-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.11.254.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2790
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786568607354921-12282-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:06:12.991349 2790 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:06:12.991551 2790 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:06:12.996646 2805 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:06:12.996716 2790 server_base.cc:1034] running on GCE node
W20260812 21:06:12.996696 2803 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:06:12.996734 2801 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:06:12.997146 2790 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:06:12.997485 2790 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:06:12.998639 2790 hybrid_clock.cc:648] HybridClock initialized: now 1786568772998604 us; error 45 us; skew 500 ppm
Aug 12 21:06:12 dist-test-slave-vpvm krb5kdc[2535](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568772, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:06:13.001385 2790 init.cc:377] Logged in from keytab as kudu/127.11.254.188@KRBTEST.COM (short username kudu)
I20260812 21:06:13.001828 2790 webserver.cc:460] Webserver started at http://127.11.254.188:39523/ using document root <none> and password file <none>
I20260812 21:06:13.002116 2790 fs_manager.cc:359] Metadata directory not provided
I20260812 21:06:13.002168 2790 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:06:13.002310 2790 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:06:13.003465 2790 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786568607354921-12282-0/minicluster-data/master-2/data/instance:
uuid: "18a1589bc77448ef8635af4d62087d84"
format_stamp: "Formatted at 2026-08-12 21:06:12 on dist-test-slave-vpvm"
server_key: "e5398e19d48e49730cd4fab62e9f5ca2"
server_key_iv: "30e3e778d67177ec31257e8dc5eee008"
server_key_version: "encryptionkey@0"
I20260812 21:06:13.003777 2790 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786568607354921-12282-0/minicluster-data/master-2/wal/instance:
uuid: "18a1589bc77448ef8635af4d62087d84"
format_stamp: "Formatted at 2026-08-12 21:06:12 on dist-test-slave-vpvm"
server_key: "e5398e19d48e49730cd4fab62e9f5ca2"
server_key_iv: "30e3e778d67177ec31257e8dc5eee008"
server_key_version: "encryptionkey@0"
I20260812 21:06:13.005064 2790 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 21:06:13.005824 2819 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:06:13.006007 2790 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 21:06:13.006093 2790 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786568607354921-12282-0/minicluster-data/master-2/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786568607354921-12282-0/minicluster-data/master-2/wal
uuid: "18a1589bc77448ef8635af4d62087d84"
format_stamp: "Formatted at 2026-08-12 21:06:12 on dist-test-slave-vpvm"
server_key: "e5398e19d48e49730cd4fab62e9f5ca2"
server_key_iv: "30e3e778d67177ec31257e8dc5eee008"
server_key_version: "encryptionkey@0"
I20260812 21:06:13.006186 2790 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786568607354921-12282-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786568607354921-12282-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786568607354921-12282-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:06:13.006966 2778 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.254.188" port: 44539 } attempt: 2
I20260812 21:06:13.007375 2782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:13.007253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.188:44539 (local address 127.0.0.1:44162)
0812 21:06:13.007304 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0812 21:06:13.007344 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.188:44539: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260812 21:06:13.007743 2782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:13.007670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.188:44539 (local address 127.0.0.1:44166)
0812 21:06:13.007704 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0812 21:06:13.007728 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.188:44539: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260812 21:06:13.007829 2778 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.254.188:44539: Network error: Client connection negotiation failed: client connection to 127.11.254.188:44539: connect: Connection refused (error 111)
I20260812 21:06:13.024360 2790 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:06:13.026626 2790 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:06:13.026750 2790 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:06:13.030730 2790 rpc_server.cc:307] RPC server started. Bound to: 127.11.254.188:44539
I20260812 21:06:13.030786 2901 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.254.188:44539 every 8 connection(s)
I20260812 21:06:13.031131 2790 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786568607354921-12282-0/minicluster-data/master-2/data/info.pb
I20260812 21:06:13.032066 2904 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:06:13.033686 2904 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.254.190" port: 43331 } has no permanent_uuid. Determining permanent_uuid...
I20260812 21:06:13.038085 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 2790
I20260812 21:06:13.038197 12282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786568607354921-12282-0/minicluster-data/master-2/wal/instance
I20260812 21:06:13.038408 12282 external_mini_cluster.cc:1428] Setting key cf13a433fea4635926fed09c04b57688
Aug 12 21:06:13 dist-test-slave-vpvm krb5kdc[2535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568772, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.188@KRBTEST.COM for kudu/127.11.254.190@KRBTEST.COM
I20260812 21:06:13.043694 2666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:13.034432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37328 (local address 127.11.254.190:43331)
0812 21:06:13.034488 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0812 21:06:13.034490 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:06:13.034703 (+ 213us) server_negotiation.cc:407] Connection header received
0812 21:06:13.035158 (+ 455us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:06:13.035159 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:06:13.035168 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:06:13.035228 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:06:13.035848 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:06:13.036326 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:06:13.037351 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:06:13.037612 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:06:13.037684 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:06:13.039983 (+ 2299us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:06:13.039984 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:06:13.039985 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:06:13.039993 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:06:13.042166 (+ 2173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:06:13.042627 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:06:13.042629 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:06:13.042629 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:06:13.042700 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:06:13.042980 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:06:13.042981 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:06:13.042982 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:06:13.043100 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:06:13.043164 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0812 21:06:13.043531 (+ 367us) server_negotiation.cc:299] Negotiation successful
0812 21:06:13.043621 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260812 21:06:13.043880 2908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:13.034403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:43331 (local address 127.0.0.1:37328)
0812 21:06:13.034573 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0812 21:06:13.034619 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 21:06:13.035074 (+ 455us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:06:13.035282 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:06:13.035285 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:06:13.035539 (+ 254us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:06:13.035749 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:06:13.035753 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:06:13.036400 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:06:13.036401 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:06:13.037240 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:06:13.037243 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:06:13.037696 (+ 453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:06:13.037697 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:06:13.037801 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:06:13.038411 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:06:13.038422 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:06:13.039886 (+ 1464us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:06:13.042238 (+ 2352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:06:13.042245 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:06:13.042246 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:06:13.042547 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:06:13.042761 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:06:13.042761 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:06:13.042762 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:06:13.042907 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:06:13.043168 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:06:13.043174 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:06:13.043351 (+ 177us) client_negotiation.cc:769] Sending connection context
0812 21:06:13.043474 (+ 123us) client_negotiation.cc:240] Negotiation successful
0812 21:06:13.043639 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":73,"threads_started":1}
Aug 12 21:06:13 dist-test-slave-vpvm krb5kdc[2535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568772, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.254.188@KRBTEST.COM
I20260812 21:06:13.044842 2904 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.254.189" port: 36381 } has no permanent_uuid. Determining permanent_uuid...
I20260812 21:06:13.047652 2912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:13.038851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44178 (local address 127.11.254.188:44539)
0812 21:06:13.039023 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0812 21:06:13.039029 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 21:06:13.039056 (+ 27us) server_negotiation.cc:407] Connection header received
0812 21:06:13.039078 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:06:13.039081 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:06:13.039102 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:06:13.039174 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:06:13.039857 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:06:13.040898 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:06:13.041767 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:06:13.042264 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:06:13.042323 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:06:13.044834 (+ 2511us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:06:13.044842 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:06:13.044843 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:06:13.044850 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:06:13.046594 (+ 1744us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:06:13.046892 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:06:13.046896 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:06:13.046896 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:06:13.046951 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:06:13.047123 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:06:13.047123 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:06:13.047124 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:06:13.047265 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:06:13.047403 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0812 21:06:13.047480 (+ 77us) server_negotiation.cc:299] Negotiation successful
0812 21:06:13.047554 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":102,"threads_started":1}
Aug 12 21:06:13 dist-test-slave-vpvm krb5kdc[2535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568772, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.188@KRBTEST.COM for kudu/127.11.254.189@KRBTEST.COM
I20260812 21:06:13.051671 2663 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.254.188" port: 44539 } attempt: 2
I20260812 21:06:13.051815 2786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:13.045199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32968 (local address 127.11.254.189:36381)
0812 21:06:13.045243 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0812 21:06:13.045244 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 21:06:13.045251 (+ 7us) server_negotiation.cc:407] Connection header received
0812 21:06:13.045300 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:06:13.045301 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:06:13.045311 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:06:13.045367 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:06:13.045709 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:06:13.046410 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:06:13.047088 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:06:13.047274 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:06:13.047334 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:06:13.049082 (+ 1748us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:06:13.049083 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:06:13.049084 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:06:13.049090 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:06:13.050829 (+ 1739us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:06:13.051129 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:06:13.051130 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:06:13.051130 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:06:13.051179 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:06:13.051339 (+ 160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:06:13.051340 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:06:13.051340 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:06:13.051453 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:06:13.051509 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0812 21:06:13.051641 (+ 132us) server_negotiation.cc:299] Negotiation successful
0812 21:06:13.051719 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 21:06:13.052137 2908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:13.045131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.189:36381 (local address 127.0.0.1:32968)
0812 21:06:13.045181 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0812 21:06:13.045192 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 21:06:13.045236 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:06:13.045369 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:06:13.045370 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:06:13.045566 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:06:13.045626 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:06:13.045628 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:06:13.046503 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:06:13.046504 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:06:13.046984 (+ 480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:06:13.046986 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:06:13.047336 (+ 350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:06:13.047336 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:06:13.047404 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:06:13.047767 (+ 363us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:06:13.047772 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:06:13.049006 (+ 1234us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:06:13.050904 (+ 1898us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:06:13.050905 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:06:13.050905 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:06:13.051068 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:06:13.051231 (+ 163us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:06:13.051232 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:06:13.051232 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:06:13.051286 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:06:13.051507 (+ 221us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:06:13.051508 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:06:13.051552 (+ 44us) client_negotiation.cc:769] Sending connection context
0812 21:06:13.051611 (+ 59us) client_negotiation.cc:240] Negotiation successful
0812 21:06:13.051719 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
Aug 12 21:06:13 dist-test-slave-vpvm krb5kdc[2535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568772, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.254.190@KRBTEST.COM
I20260812 21:06:13.052656 2904 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.254.188" port: 44539 } has no permanent_uuid. Determining permanent_uuid...
I20260812 21:06:13.055692 2666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:13.048785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37334 (local address 127.11.254.190:43331)
0812 21:06:13.048844 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0812 21:06:13.048846 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:06:13.048855 (+ 9us) server_negotiation.cc:407] Connection header received
0812 21:06:13.048877 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:06:13.048879 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:06:13.048888 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:06:13.048940 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:06:13.049362 (+ 422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:06:13.049849 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:06:13.050515 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:06:13.050717 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:06:13.050770 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:06:13.052949 (+ 2179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:06:13.052950 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:06:13.052950 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:06:13.052957 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:06:13.054538 (+ 1581us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:06:13.054895 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:06:13.054896 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:06:13.054896 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:06:13.054968 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:06:13.055132 (+ 164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:06:13.055134 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:06:13.055134 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:06:13.055243 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:06:13.055436 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0812 21:06:13.055475 (+ 39us) server_negotiation.cc:299] Negotiation successful
0812 21:06:13.055552 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
Aug 12 21:06:13 dist-test-slave-vpvm krb5kdc[2535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568772, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.188@KRBTEST.COM for kudu/127.11.254.188@KRBTEST.COM
Aug 12 21:06:13 dist-test-slave-vpvm krb5kdc[2535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568772, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.190@KRBTEST.COM for kudu/127.11.254.188@KRBTEST.COM
I20260812 21:06:13.059374 2908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:13.052879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.188:44539 (local address 127.0.0.1:44202)
0812 21:06:13.052909 (+ 30us) negotiation.cc:107] Waiting for socket to connect
0812 21:06:13.052924 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 21:06:13.052959 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:06:13.053128 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:06:13.053129 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:06:13.053246 (+ 117us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:06:13.053270 (+ 24us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:06:13.053271 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:06:13.053865 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:06:13.053865 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:06:13.054376 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:06:13.054377 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:06:13.054763 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:06:13.054764 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:06:13.054816 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:06:13.055167 (+ 351us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:06:13.055172 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:06:13.056496 (+ 1324us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:06:13.058264 (+ 1768us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:06:13.058265 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:06:13.058265 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:06:13.058513 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:06:13.058716 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:06:13.058717 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:06:13.058717 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:06:13.058762 (+ 45us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:06:13.058995 (+ 233us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:06:13.058996 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:06:13.059048 (+ 52us) client_negotiation.cc:769] Sending connection context
0812 21:06:13.059134 (+ 86us) client_negotiation.cc:240] Negotiation successful
0812 21:06:13.059274 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17}
I20260812 21:06:13.059625 2913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:13.052905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44202 (local address 127.11.254.188:44539)
0812 21:06:13.053038 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0812 21:06:13.053039 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 21:06:13.053044 (+ 5us) server_negotiation.cc:407] Connection header received
0812 21:06:13.053058 (+ 14us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:06:13.053058 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:06:13.053066 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:06:13.053116 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:06:13.053338 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:06:13.053801 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:06:13.054444 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:06:13.054696 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:06:13.054761 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:06:13.056573 (+ 1812us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:06:13.056575 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:06:13.056575 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:06:13.056581 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:06:13.058188 (+ 1607us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:06:13.058585 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:06:13.058586 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:06:13.058586 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:06:13.058647 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:06:13.058812 (+ 165us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:06:13.058813 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:06:13.058813 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:06:13.058924 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:06:13.058991 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 21:06:13.059179 (+ 188us) server_negotiation.cc:299] Negotiation successful
0812 21:06:13.059415 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":87,"server-negotiator.queue_time_us":118,"thread_start_us":90,"threads_started":1}
I20260812 21:06:13.060693 2665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:13.051942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.188:44539 (local address 127.0.0.1:44186)
0812 21:06:13.052033 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0812 21:06:13.052042 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 21:06:13.052077 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:06:13.052239 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:06:13.052241 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:06:13.052390 (+ 149us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:06:13.052429 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:06:13.052431 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:06:13.053540 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:06:13.053541 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:06:13.054526 (+ 985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:06:13.054529 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:06:13.055203 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:06:13.055204 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:06:13.055265 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:06:13.055607 (+ 342us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:06:13.055611 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:06:13.057150 (+ 1539us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:06:13.059588 (+ 2438us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:06:13.059589 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:06:13.059590 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:06:13.059808 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:06:13.060014 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:06:13.060015 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:06:13.060016 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:06:13.060091 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:06:13.060361 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:06:13.060362 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:06:13.060409 (+ 47us) client_negotiation.cc:769] Sending connection context
0812 21:06:13.060486 (+ 77us) client_negotiation.cc:240] Negotiation successful
0812 21:06:13.060597 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":75}
I20260812 21:06:13.060698 2912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:13.052058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44186 (local address 127.11.254.188:44539)
0812 21:06:13.052096 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0812 21:06:13.052098 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:06:13.052105 (+ 7us) server_negotiation.cc:407] Connection header received
0812 21:06:13.052133 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:06:13.052134 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:06:13.052143 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:06:13.052498 (+ 355us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:06:13.052531 (+ 33us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:06:13.053448 (+ 917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:06:13.054624 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:06:13.054872 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:06:13.056161 (+ 1289us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:06:13.057237 (+ 1076us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:06:13.057238 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:06:13.057238 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:06:13.057245 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:06:13.059499 (+ 2254us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:06:13.059890 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:06:13.059891 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:06:13.059891 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:06:13.059947 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:06:13.060155 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:06:13.060156 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:06:13.060156 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:06:13.060301 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:06:13.060367 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0812 21:06:13.060532 (+ 165us) server_negotiation.cc:299] Negotiation successful
0812 21:06:13.060624 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 21:06:13.061735 2904 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 18a1589bc77448ef8635af4d62087d84: Bootstrap starting.
Aug 12 21:06:13 dist-test-slave-vpvm krb5kdc[2535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568772, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.254.189@KRBTEST.COM
I20260812 21:06:13.062177 2663 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eeab60837d1942369667c7c668515091: Bootstrap starting.
I20260812 21:06:13.062582 2904 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 18a1589bc77448ef8635af4d62087d84: Neither blocks nor log segments found. Creating new log.
I20260812 21:06:13.062862 2663 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eeab60837d1942369667c7c668515091: Neither blocks nor log segments found. Creating new log.
I20260812 21:06:13.062896 2904 log.cc:824] T 00000000000000000000000000000000 P 18a1589bc77448ef8635af4d62087d84: Log is configured to *not* fsync() on all Append() calls
I20260812 21:06:13.063163 2663 log.cc:824] T 00000000000000000000000000000000 P eeab60837d1942369667c7c668515091: Log is configured to *not* fsync() on all Append() calls
I20260812 21:06:13.063894 2904 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 18a1589bc77448ef8635af4d62087d84: No bootstrap required, opened a new log
I20260812 21:06:13.063928 2663 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eeab60837d1942369667c7c668515091: No bootstrap required, opened a new log
I20260812 21:06:13.065537 2786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:13.057439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32984 (local address 127.11.254.189:36381)
0812 21:06:13.057476 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0812 21:06:13.057477 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 21:06:13.057485 (+ 8us) server_negotiation.cc:407] Connection header received
0812 21:06:13.057553 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:06:13.057555 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:06:13.057566 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:06:13.057625 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:06:13.058263 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:06:13.058826 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:06:13.059702 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:06:13.059970 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:06:13.060012 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:06:13.062394 (+ 2382us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:06:13.062395 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:06:13.062396 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:06:13.062404 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:06:13.063951 (+ 1547us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:06:13.064332 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:06:13.064333 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:06:13.064334 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:06:13.064411 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:06:13.064628 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:06:13.064630 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:06:13.064630 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:06:13.064773 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:06:13.064970 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0812 21:06:13.065357 (+ 387us) server_negotiation.cc:299] Negotiation successful
0812 21:06:13.065444 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 21:06:13.065570 2904 raft_consensus.cc:348] T 00000000000000000000000000000000 P 18a1589bc77448ef8635af4d62087d84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eeab60837d1942369667c7c668515091" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 43331 } } peers { permanent_uuid: "1c1144d3d59c4213b01603eef88e9932" member_type: VOTER last_known_addr { host: "127.11.254.189" port: 36381 } } peers { permanent_uuid: "18a1589bc77448ef8635af4d62087d84" member_type: VOTER last_known_addr { host: "127.11.254.188" port: 44539 } }
I20260812 21:06:13.065743 2904 raft_consensus.cc:374] T 00000000000000000000000000000000 P 18a1589bc77448ef8635af4d62087d84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:06:13.065793 2904 raft_consensus.cc:729] T 00000000000000000000000000000000 P 18a1589bc77448ef8635af4d62087d84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18a1589bc77448ef8635af4d62087d84, State: Initialized, Role: FOLLOWER
I20260812 21:06:13.066007 2663 raft_consensus.cc:348] T 00000000000000000000000000000000 P eeab60837d1942369667c7c668515091 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eeab60837d1942369667c7c668515091" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 43331 } } peers { permanent_uuid: "1c1144d3d59c4213b01603eef88e9932" member_type: VOTER last_known_addr { host: "127.11.254.189" port: 36381 } } peers { permanent_uuid: "18a1589bc77448ef8635af4d62087d84" member_type: VOTER last_known_addr { host: "127.11.254.188" port: 44539 } }
I20260812 21:06:13.066136 2663 raft_consensus.cc:374] T 00000000000000000000000000000000 P eeab60837d1942369667c7c668515091 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:06:13.066182 2663 raft_consensus.cc:729] T 00000000000000000000000000000000 P eeab60837d1942369667c7c668515091 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eeab60837d1942369667c7c668515091, State: Initialized, Role: FOLLOWER
I20260812 21:06:13.066275 2904 consensus_queue.cc:260] T 00000000000000000000000000000000 P 18a1589bc77448ef8635af4d62087d84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eeab60837d1942369667c7c668515091" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 43331 } } peers { permanent_uuid: "1c1144d3d59c4213b01603eef88e9932" member_type: VOTER last_known_addr { host: "127.11.254.189" port: 36381 } } peers { permanent_uuid: "18a1589bc77448ef8635af4d62087d84" member_type: VOTER last_known_addr { host: "127.11.254.188" port: 44539 } }
I20260812 21:06:13.066300 2663 consensus_queue.cc:260] T 00000000000000000000000000000000 P eeab60837d1942369667c7c668515091 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eeab60837d1942369667c7c668515091" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 43331 } } peers { permanent_uuid: "1c1144d3d59c4213b01603eef88e9932" member_type: VOTER last_known_addr { host: "127.11.254.189" port: 36381 } } peers { permanent_uuid: "18a1589bc77448ef8635af4d62087d84" member_type: VOTER last_known_addr { host: "127.11.254.188" port: 44539 } }
I20260812 21:06:13.066622 2919 sys_catalog.cc:455] T 00000000000000000000000000000000 P 18a1589bc77448ef8635af4d62087d84 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eeab60837d1942369667c7c668515091" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 43331 } } peers { permanent_uuid: "1c1144d3d59c4213b01603eef88e9932" member_type: VOTER last_known_addr { host: "127.11.254.189" port: 36381 } } peers { permanent_uuid: "18a1589bc77448ef8635af4d62087d84" member_type: VOTER last_known_addr { host: "127.11.254.188" port: 44539 } } }
I20260812 21:06:13.066692 2663 sys_catalog.cc:564] T 00000000000000000000000000000000 P eeab60837d1942369667c7c668515091 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 21:06:13.066807 2919 sys_catalog.cc:458] T 00000000000000000000000000000000 P 18a1589bc77448ef8635af4d62087d84 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 21:06:13.066814 2921 sys_catalog.cc:455] T 00000000000000000000000000000000 P eeab60837d1942369667c7c668515091 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eeab60837d1942369667c7c668515091" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 43331 } } peers { permanent_uuid: "1c1144d3d59c4213b01603eef88e9932" member_type: VOTER last_known_addr { host: "127.11.254.189" port: 36381 } } peers { permanent_uuid: "18a1589bc77448ef8635af4d62087d84" member_type: VOTER last_known_addr { host: "127.11.254.188" port: 44539 } } }
I20260812 21:06:13.066682 2904 sys_catalog.cc:564] T 00000000000000000000000000000000 P 18a1589bc77448ef8635af4d62087d84 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 21:06:13.066980 2921 sys_catalog.cc:458] T 00000000000000000000000000000000 P eeab60837d1942369667c7c668515091 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 21:06:13.067167 12282 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260812 21:06:13.068459 2936 catalog_manager.cc:1558] T 00000000000000000000000000000000 P eeab60837d1942369667c7c668515091: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 21:06:13.068569 2936 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260812 21:06:13.070091 2960 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 18a1589bc77448ef8635af4d62087d84: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 21:06:13.070214 2960 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 21:06:13.069293 2558 master_runner.cc:186] Error getting master registration for 127.11.254.189:36381: OK, instance_id { permanent_uuid: "1c1144d3d59c4213b01603eef88e9932" instance_seqno: 1786568772865754 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 21:06:13.071381 2558 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 21:06:13.071395 2790 master_runner.cc:186] Error getting master registration for 127.11.254.189:36381: OK, instance_id { permanent_uuid: "1c1144d3d59c4213b01603eef88e9932" instance_seqno: 1786568772865754 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 21:06:13.074326 2790 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 21:06:13.079589 2786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:13.069261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32990 (local address 127.11.254.189:36381)
0812 21:06:13.069305 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0812 21:06:13.069307 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:06:13.071716 (+ 2409us) server_negotiation.cc:407] Connection header received
0812 21:06:13.071818 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:06:13.071820 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:06:13.071832 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:06:13.071861 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:06:13.072826 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:06:13.073537 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:06:13.074688 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:06:13.074986 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:06:13.075059 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:06:13.076588 (+ 1529us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:06:13.076590 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:06:13.076590 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:06:13.076598 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:06:13.078295 (+ 1697us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:06:13.078714 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:06:13.078715 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:06:13.078715 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:06:13.078778 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:06:13.078982 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:06:13.078983 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:06:13.078983 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:06:13.079119 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:06:13.079197 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0812 21:06:13.079403 (+ 206us) server_negotiation.cc:299] Negotiation successful
0812 21:06:13.079488 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 21:06:13.083281 2912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:13.072100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44206 (local address 127.11.254.188:44539)
0812 21:06:13.072140 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0812 21:06:13.072142 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:06:13.074935 (+ 2793us) server_negotiation.cc:407] Connection header received
0812 21:06:13.075026 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:06:13.075027 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:06:13.075038 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:06:13.075065 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:06:13.076163 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:06:13.077163 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:06:13.078425 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:06:13.078719 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:06:13.078762 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:06:13.080456 (+ 1694us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:06:13.080457 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:06:13.080458 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:06:13.080465 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:06:13.082026 (+ 1561us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:06:13.082403 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:06:13.082404 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:06:13.082404 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:06:13.082460 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:06:13.082663 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:06:13.082664 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:06:13.082664 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:06:13.082798 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:06:13.082869 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0812 21:06:13.083079 (+ 210us) server_negotiation.cc:299] Negotiation successful
0812 21:06:13.083182 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 21:06:13.087725 2666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:13.071781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37340 (local address 127.11.254.190:43331)
0812 21:06:13.071820 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0812 21:06:13.071821 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 21:06:13.074780 (+ 2959us) server_negotiation.cc:407] Connection header received
0812 21:06:13.074845 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:06:13.074847 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:06:13.074859 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:06:13.075009 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:06:13.080640 (+ 5631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:06:13.081831 (+ 1191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:06:13.082868 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:06:13.083176 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:06:13.083219 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:06:13.084936 (+ 1717us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:06:13.084938 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:06:13.084939 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:06:13.084946 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:06:13.086533 (+ 1587us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:06:13.086927 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:06:13.086928 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:06:13.086928 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:06:13.086988 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:06:13.087174 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:06:13.087175 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:06:13.087175 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:06:13.087311 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:06:13.087401 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0812 21:06:13.087537 (+ 136us) server_negotiation.cc:299] Negotiation successful
0812 21:06:13.087626 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 21:06:13.098984 2778 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.254.188" port: 44539 } attempt: 3
Aug 12 21:06:13 dist-test-slave-vpvm krb5kdc[2535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568772, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.189@KRBTEST.COM for kudu/127.11.254.188@KRBTEST.COM
I20260812 21:06:13.107566 2912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:13.099395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44220 (local address 127.11.254.188:44539)
0812 21:06:13.099440 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0812 21:06:13.099442 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:06:13.099451 (+ 9us) server_negotiation.cc:407] Connection header received
0812 21:06:13.099531 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:06:13.099532 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:06:13.099543 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:06:13.099599 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:06:13.100028 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:06:13.100784 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:06:13.101794 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:06:13.102131 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:06:13.102193 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:06:13.104611 (+ 2418us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:06:13.104612 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:06:13.104613 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:06:13.104622 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:06:13.106317 (+ 1695us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:06:13.106774 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:06:13.106775 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:06:13.106776 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:06:13.106841 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:06:13.107023 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:06:13.107023 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:06:13.107024 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:06:13.107167 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:06:13.107230 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0812 21:06:13.107376 (+ 146us) server_negotiation.cc:299] Negotiation successful
0812 21:06:13.107464 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 21:06:13.107570 2782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:13.099313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.188:44539 (local address 127.0.0.1:44220)
0812 21:06:13.099392 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0812 21:06:13.099407 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 21:06:13.099464 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:06:13.099612 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:06:13.099613 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:06:13.099841 (+ 228us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:06:13.099915 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:06:13.099918 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:06:13.100868 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:06:13.100869 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:06:13.101704 (+ 835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:06:13.101707 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:06:13.102263 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:06:13.102264 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:06:13.102352 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:06:13.102912 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:06:13.102918 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:06:13.104506 (+ 1588us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:06:13.106445 (+ 1939us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:06:13.106446 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:06:13.106447 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:06:13.106692 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:06:13.106900 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:06:13.106901 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:06:13.106901 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:06:13.106967 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:06:13.107225 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:06:13.107226 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:06:13.107279 (+ 53us) client_negotiation.cc:769] Sending connection context
0812 21:06:13.107347 (+ 68us) client_negotiation.cc:240] Negotiation successful
0812 21:06:13.107462 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260812 21:06:13.109668 2778 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1c1144d3d59c4213b01603eef88e9932: Bootstrap starting.
I20260812 21:06:13.110467 2778 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1c1144d3d59c4213b01603eef88e9932: Neither blocks nor log segments found. Creating new log.
I20260812 21:06:13.110791 2778 log.cc:824] T 00000000000000000000000000000000 P 1c1144d3d59c4213b01603eef88e9932: Log is configured to *not* fsync() on all Append() calls
I20260812 21:06:13.111721 2778 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1c1144d3d59c4213b01603eef88e9932: No bootstrap required, opened a new log
I20260812 21:06:13.113808 2778 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1c1144d3d59c4213b01603eef88e9932 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eeab60837d1942369667c7c668515091" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 43331 } } peers { permanent_uuid: "1c1144d3d59c4213b01603eef88e9932" member_type: VOTER last_known_addr { host: "127.11.254.189" port: 36381 } } peers { permanent_uuid: "18a1589bc77448ef8635af4d62087d84" member_type: VOTER last_known_addr { host: "127.11.254.188" port: 44539 } }
I20260812 21:06:13.114046 2778 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1c1144d3d59c4213b01603eef88e9932 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:06:13.114130 2778 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1c1144d3d59c4213b01603eef88e9932 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c1144d3d59c4213b01603eef88e9932, State: Initialized, Role: FOLLOWER
I20260812 21:06:13.114316 2778 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1c1144d3d59c4213b01603eef88e9932 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eeab60837d1942369667c7c668515091" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 43331 } } peers { permanent_uuid: "1c1144d3d59c4213b01603eef88e9932" member_type: VOTER last_known_addr { host: "127.11.254.189" port: 36381 } } peers { permanent_uuid: "18a1589bc77448ef8635af4d62087d84" member_type: VOTER last_known_addr { host: "127.11.254.188" port: 44539 } }
I20260812 21:06:13.114682 2970 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c1144d3d59c4213b01603eef88e9932 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eeab60837d1942369667c7c668515091" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 43331 } } peers { permanent_uuid: "1c1144d3d59c4213b01603eef88e9932" member_type: VOTER last_known_addr { host: "127.11.254.189" port: 36381 } } peers { permanent_uuid: "18a1589bc77448ef8635af4d62087d84" member_type: VOTER last_known_addr { host: "127.11.254.188" port: 44539 } } }
I20260812 21:06:13.114826 2970 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c1144d3d59c4213b01603eef88e9932 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 21:06:13.114740 2778 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1c1144d3d59c4213b01603eef88e9932 [sys.catalog]: configured and running, proceeding with master startup.
W20260812 21:06:13.116760 2989 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1c1144d3d59c4213b01603eef88e9932: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 21:06:13.116873 2989 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 21:06:13.118990 2676 master_runner.cc:263] No leader master found from master 1c1144d3d59c4213b01603eef88e9932
I20260812 21:06:13.119040 2676 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 21:06:13.127220 2919 raft_consensus.cc:482] T 00000000000000000000000000000000 P 18a1589bc77448ef8635af4d62087d84 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 21:06:13.127385 2919 raft_consensus.cc:504] T 00000000000000000000000000000000 P 18a1589bc77448ef8635af4d62087d84 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eeab60837d1942369667c7c668515091" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 43331 } } peers { permanent_uuid: "1c1144d3d59c4213b01603eef88e9932" member_type: VOTER last_known_addr { host: "127.11.254.189" port: 36381 } } peers { permanent_uuid: "18a1589bc77448ef8635af4d62087d84" member_type: VOTER last_known_addr { host: "127.11.254.188" port: 44539 } }
I20260812 21:06:13.127665 2919 leader_election.cc:288] T 00000000000000000000000000000000 P 18a1589bc77448ef8635af4d62087d84 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eeab60837d1942369667c7c668515091 (127.11.254.190:43331), 1c1144d3d59c4213b01603eef88e9932 (127.11.254.189:36381)
I20260812 21:06:13.128240 2620 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "18a1589bc77448ef8635af4d62087d84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eeab60837d1942369667c7c668515091" is_pre_election: true
I20260812 21:06:13.128408 2620 raft_consensus.cc:2454] T 00000000000000000000000000000000 P eeab60837d1942369667c7c668515091 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 18a1589bc77448ef8635af4d62087d84 in term 0.
I20260812 21:06:13.128750 2823 leader_election.cc:302] T 00000000000000000000000000000000 P 18a1589bc77448ef8635af4d62087d84 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18a1589bc77448ef8635af4d62087d84, eeab60837d1942369667c7c668515091; no voters:
I20260812 21:06:13.128970 2919 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 18a1589bc77448ef8635af4d62087d84 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 21:06:13.129024 2919 raft_consensus.cc:482] T 00000000000000000000000000000000 P 18a1589bc77448ef8635af4d62087d84 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 21:06:13.129065 2919 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 18a1589bc77448ef8635af4d62087d84 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:06:13.130107 2919 raft_consensus.cc:504] T 00000000000000000000000000000000 P 18a1589bc77448ef8635af4d62087d84 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eeab60837d1942369667c7c668515091" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 43331 } } peers { permanent_uuid: "1c1144d3d59c4213b01603eef88e9932" member_type: VOTER last_known_addr { host: "127.11.254.189" port: 36381 } } peers { permanent_uuid: "18a1589bc77448ef8635af4d62087d84" member_type: VOTER last_known_addr { host: "127.11.254.188" port: 44539 } }
I20260812 21:06:13.130250 2919 leader_election.cc:288] T 00000000000000000000000000000000 P 18a1589bc77448ef8635af4d62087d84 [CANDIDATE]: Term 1 election: Requested vote from peers eeab60837d1942369667c7c668515091 (127.11.254.190:43331), 1c1144d3d59c4213b01603eef88e9932 (127.11.254.189:36381)
I20260812 21:06:13.131278 2620 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "18a1589bc77448ef8635af4d62087d84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eeab60837d1942369667c7c668515091"
I20260812 21:06:13.131363 2620 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eeab60837d1942369667c7c668515091 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:06:13.132247 2620 raft_consensus.cc:2454] T 00000000000000000000000000000000 P eeab60837d1942369667c7c668515091 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 18a1589bc77448ef8635af4d62087d84 in term 1.
I20260812 21:06:13.132529 2823 leader_election.cc:302] T 00000000000000000000000000000000 P 18a1589bc77448ef8635af4d62087d84 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18a1589bc77448ef8635af4d62087d84, eeab60837d1942369667c7c668515091; no voters:
I20260812 21:06:13.133255 2741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "18a1589bc77448ef8635af4d62087d84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c1144d3d59c4213b01603eef88e9932" is_pre_election: true
I20260812 21:06:13.133431 2741 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1c1144d3d59c4213b01603eef88e9932 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 18a1589bc77448ef8635af4d62087d84 in term 0.
I20260812 21:06:13.133543 2740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "18a1589bc77448ef8635af4d62087d84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c1144d3d59c4213b01603eef88e9932"
I20260812 21:06:13.133592 2740 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1c1144d3d59c4213b01603eef88e9932 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:06:13.134850 2740 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1c1144d3d59c4213b01603eef88e9932 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 18a1589bc77448ef8635af4d62087d84 in term 1.
I20260812 21:06:13.135720 2919 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 18a1589bc77448ef8635af4d62087d84 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:06:13.135914 2919 raft_consensus.cc:686] T 00000000000000000000000000000000 P 18a1589bc77448ef8635af4d62087d84 [term 1 LEADER]: Becoming Leader. State: Replica: 18a1589bc77448ef8635af4d62087d84, State: Running, Role: LEADER
I20260812 21:06:13.136004 2919 consensus_queue.cc:237] T 00000000000000000000000000000000 P 18a1589bc77448ef8635af4d62087d84 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eeab60837d1942369667c7c668515091" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 43331 } } peers { permanent_uuid: "1c1144d3d59c4213b01603eef88e9932" member_type: VOTER last_known_addr { host: "127.11.254.189" port: 36381 } } peers { permanent_uuid: "18a1589bc77448ef8635af4d62087d84" member_type: VOTER last_known_addr { host: "127.11.254.188" port: 44539 } }
I20260812 21:06:13.136384 2919 sys_catalog.cc:455] T 00000000000000000000000000000000 P 18a1589bc77448ef8635af4d62087d84 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 18a1589bc77448ef8635af4d62087d84. Latest consensus state: current_term: 1 leader_uuid: "18a1589bc77448ef8635af4d62087d84" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eeab60837d1942369667c7c668515091" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 43331 } } peers { permanent_uuid: "1c1144d3d59c4213b01603eef88e9932" member_type: VOTER last_known_addr { host: "127.11.254.189" port: 36381 } } peers { permanent_uuid: "18a1589bc77448ef8635af4d62087d84" member_type: VOTER last_known_addr { host: "127.11.254.188" port: 44539 } } }
I20260812 21:06:13.136443 2919 sys_catalog.cc:458] T 00000000000000000000000000000000 P 18a1589bc77448ef8635af4d62087d84 [sys.catalog]: This master's current role is: LEADER
I20260812 21:06:13.137315 2993 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 21:06:13.137543 2993 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 21:06:13.139341 2740 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1c1144d3d59c4213b01603eef88e9932 [term 1 FOLLOWER]: Refusing update from remote peer 18a1589bc77448ef8635af4d62087d84: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 21:06:13.139763 2991 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 18a1589bc77448ef8635af4d62087d84 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c1144d3d59c4213b01603eef88e9932" member_type: VOTER last_known_addr { host: "127.11.254.189" port: 36381 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 21:06:13.141208 2970 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c1144d3d59c4213b01603eef88e9932 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 18a1589bc77448ef8635af4d62087d84. Latest consensus state: current_term: 1 leader_uuid: "18a1589bc77448ef8635af4d62087d84" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eeab60837d1942369667c7c668515091" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 43331 } } peers { permanent_uuid: "1c1144d3d59c4213b01603eef88e9932" member_type: VOTER last_known_addr { host: "127.11.254.189" port: 36381 } } peers { permanent_uuid: "18a1589bc77448ef8635af4d62087d84" member_type: VOTER last_known_addr { host: "127.11.254.188" port: 44539 } } }
I20260812 21:06:13.141320 2970 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c1144d3d59c4213b01603eef88e9932 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 21:06:13.141546 2620 raft_consensus.cc:1261] T 00000000000000000000000000000000 P eeab60837d1942369667c7c668515091 [term 1 FOLLOWER]: Refusing update from remote peer 18a1589bc77448ef8635af4d62087d84: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 21:06:13.141845 2991 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 18a1589bc77448ef8635af4d62087d84 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eeab60837d1942369667c7c668515091" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 43331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 21:06:13.143108 2921 sys_catalog.cc:455] T 00000000000000000000000000000000 P eeab60837d1942369667c7c668515091 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 18a1589bc77448ef8635af4d62087d84. Latest consensus state: current_term: 1 leader_uuid: "18a1589bc77448ef8635af4d62087d84" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eeab60837d1942369667c7c668515091" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 43331 } } peers { permanent_uuid: "1c1144d3d59c4213b01603eef88e9932" member_type: VOTER last_known_addr { host: "127.11.254.189" port: 36381 } } peers { permanent_uuid: "18a1589bc77448ef8635af4d62087d84" member_type: VOTER last_known_addr { host: "127.11.254.188" port: 44539 } } }
I20260812 21:06:13.143244 2921 sys_catalog.cc:458] T 00000000000000000000000000000000 P eeab60837d1942369667c7c668515091 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 21:06:13.144469 2991 sys_catalog.cc:455] T 00000000000000000000000000000000 P 18a1589bc77448ef8635af4d62087d84 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "18a1589bc77448ef8635af4d62087d84" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eeab60837d1942369667c7c668515091" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 43331 } } peers { permanent_uuid: "1c1144d3d59c4213b01603eef88e9932" member_type: VOTER last_known_addr { host: "127.11.254.189" port: 36381 } } peers { permanent_uuid: "18a1589bc77448ef8635af4d62087d84" member_type: VOTER last_known_addr { host: "127.11.254.188" port: 44539 } } }
I20260812 21:06:13.144568 2991 sys_catalog.cc:458] T 00000000000000000000000000000000 P 18a1589bc77448ef8635af4d62087d84 [sys.catalog]: This master's current role is: LEADER
I20260812 21:06:13.145143 2970 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c1144d3d59c4213b01603eef88e9932 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "18a1589bc77448ef8635af4d62087d84" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eeab60837d1942369667c7c668515091" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 43331 } } peers { permanent_uuid: "1c1144d3d59c4213b01603eef88e9932" member_type: VOTER last_known_addr { host: "127.11.254.189" port: 36381 } } peers { permanent_uuid: "18a1589bc77448ef8635af4d62087d84" member_type: VOTER last_known_addr { host: "127.11.254.188" port: 44539 } } }
I20260812 21:06:13.145242 2970 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c1144d3d59c4213b01603eef88e9932 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 21:06:13.145303 2995 mvcc.cc:205] Tried to move back new op lower bound from 7317785694775832576 to 7317785694765658112. Current Snapshot: MvccSnapshot[applied={T|T < 7317785694775832576}]
I20260812 21:06:13.145426 2999 mvcc.cc:205] Tried to move back new op lower bound from 7317785694775832576 to 7317785694765658112. Current Snapshot: MvccSnapshot[applied={T|T < 7317785694775832576}]
I20260812 21:06:13.147166 2993 catalog_manager.cc:1347] Generated new cluster ID: da1059e231704ffbabc43e3e23795bdb
I20260812 21:06:13.147228 2993 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 21:06:13.151160 2919 sys_catalog.cc:455] T 00000000000000000000000000000000 P 18a1589bc77448ef8635af4d62087d84 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "18a1589bc77448ef8635af4d62087d84" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eeab60837d1942369667c7c668515091" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 43331 } } peers { permanent_uuid: "1c1144d3d59c4213b01603eef88e9932" member_type: VOTER last_known_addr { host: "127.11.254.189" port: 36381 } } peers { permanent_uuid: "18a1589bc77448ef8635af4d62087d84" member_type: VOTER last_known_addr { host: "127.11.254.188" port: 44539 } } }
I20260812 21:06:13.151283 2919 sys_catalog.cc:458] T 00000000000000000000000000000000 P 18a1589bc77448ef8635af4d62087d84 [sys.catalog]: This master's current role is: LEADER
I20260812 21:06:13.151794 2921 sys_catalog.cc:455] T 00000000000000000000000000000000 P eeab60837d1942369667c7c668515091 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "18a1589bc77448ef8635af4d62087d84" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eeab60837d1942369667c7c668515091" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 43331 } } peers { permanent_uuid: "1c1144d3d59c4213b01603eef88e9932" member_type: VOTER last_known_addr { host: "127.11.254.189" port: 36381 } } peers { permanent_uuid: "18a1589bc77448ef8635af4d62087d84" member_type: VOTER last_known_addr { host: "127.11.254.188" port: 44539 } } }
I20260812 21:06:13.151896 2921 sys_catalog.cc:458] T 00000000000000000000000000000000 P eeab60837d1942369667c7c668515091 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 21:06:13.151985 3002 mvcc.cc:205] Tried to move back new op lower bound from 7317785694775832576 to 7317785694765658112. Current Snapshot: MvccSnapshot[applied={T|T < 7317785694775832576}]
I20260812 21:06:13.165349 2993 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 21:06:13.166181 2993 catalog_manager.cc:1504] Loading token signing keys...
I20260812 21:06:13.183022 2993 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 18a1589bc77448ef8635af4d62087d84: Generated new TSK 0
I20260812 21:06:13.183777 2993 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 21:06:14.069144 2936 catalog_manager.cc:1259] Loaded cluster ID: da1059e231704ffbabc43e3e23795bdb
I20260812 21:06:14.069224 2936 catalog_manager.cc:1552] T 00000000000000000000000000000000 P eeab60837d1942369667c7c668515091: loading cluster ID for follower catalog manager: success
I20260812 21:06:14.070104 2936 catalog_manager.cc:1574] T 00000000000000000000000000000000 P eeab60837d1942369667c7c668515091: acquiring CA information for follower catalog manager: success
I20260812 21:06:14.070425 2936 catalog_manager.cc:1602] T 00000000000000000000000000000000 P eeab60837d1942369667c7c668515091: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 21:06:14.117358 2989 catalog_manager.cc:1259] Loaded cluster ID: da1059e231704ffbabc43e3e23795bdb
I20260812 21:06:14.117441 2989 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 1c1144d3d59c4213b01603eef88e9932: loading cluster ID for follower catalog manager: success
I20260812 21:06:14.118314 2989 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 1c1144d3d59c4213b01603eef88e9932: acquiring CA information for follower catalog manager: success
I20260812 21:06:14.118660 2989 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 1c1144d3d59c4213b01603eef88e9932: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 21:06:14.242399 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 2558
I20260812 21:06:14.260399 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 2676
I20260812 21:06:14.271816 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 2790
2026-08-12T21:06:14Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3036 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786568607354921-12282-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 21:06:14 dist-test-slave-vpvm krb5kdc[3022](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 21:06:14 dist-test-slave-vpvm krb5kdc[3022](info): set up 2 sockets
Aug 12 21:06:14 dist-test-slave-vpvm krb5kdc[3022](info): commencing operation
krb5kdc: starting...
W20260812 21:06:15.455459 12282 mini_kdc.cc:121] Time spent starting KDC: real 1.154s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786568607354921-12282-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786568607354921-12282-0/krb5kdc/test-user.keytab.
Aug 12 21:06:15 dist-test-slave-vpvm krb5kdc[3022](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568775, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T21:06:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T21:06:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.254.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.254.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.254.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.190.keytab.
Entry for principal kudu/127.11.254.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.190.keytab.
I20260812 21:06:15.595345 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786568607354921-12282-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786568607354921-12282-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786568607354921-12282-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.254.190:40331
--webserver_interface=127.11.254.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.190.keytab
--principal=kudu/127.11.254.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.254.148:46499
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.254.190:40331
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786568607354921-12282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786568607354921-12282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786568607354921-12282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:06:15.678539 3048 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:06:15.678781 3048 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:06:15.678817 3048 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:06:15.679991 3048 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 21:06:15.680034 3048 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:06:15.680052 3048 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:06:15.680068 3048 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 21:06:15.680084 3048 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 21:06:15.681623 3048 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.254.148:46499
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786568607354921-12282-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786568607354921-12282-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.254.190:40331
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.190.keytab
--principal=kudu/127.11.254.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.254.190:40331
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.254.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3048
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786568607354921-12282-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:06:15.681867 3048 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:06:15.682157 3048 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:06:15.687366 3056 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:06:15.687387 3048 server_base.cc:1034] running on GCE node
W20260812 21:06:15.687389 3054 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:06:15.687368 3058 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:06:15.687872 3048 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:06:15.688174 3048 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:06:15.689313 3048 hybrid_clock.cc:648] HybridClock initialized: now 1786568775689291 us; error 37 us; skew 500 ppm
Aug 12 21:06:15 dist-test-slave-vpvm krb5kdc[3022](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568775, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:06:15.691972 3048 init.cc:377] Logged in from keytab as kudu/127.11.254.190@KRBTEST.COM (short username kudu)
I20260812 21:06:15.692427 3048 webserver.cc:460] Webserver started at http://127.11.254.190:42615/ using document root <none> and password file <none>
I20260812 21:06:15.692672 3048 fs_manager.cc:359] Metadata directory not provided
I20260812 21:06:15.692723 3048 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:06:15.692850 3048 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:06:15.694042 3048 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786568607354921-12282-0/minicluster-data/master-0/data/instance:
uuid: "fef72df91dce4bd7a117a04d0ee76176"
format_stamp: "Formatted at 2026-08-12 21:06:15 on dist-test-slave-vpvm"
server_key: "2dafaf8f26615ae14d755d00ee18a140"
server_key_iv: "72cfcf167f10ee4fa2e98ba82c9dab5d"
server_key_version: "encryptionkey@0"
I20260812 21:06:15.694403 3048 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786568607354921-12282-0/minicluster-data/master-0/wal/instance:
uuid: "fef72df91dce4bd7a117a04d0ee76176"
format_stamp: "Formatted at 2026-08-12 21:06:15 on dist-test-slave-vpvm"
server_key: "2dafaf8f26615ae14d755d00ee18a140"
server_key_iv: "72cfcf167f10ee4fa2e98ba82c9dab5d"
server_key_version: "encryptionkey@0"
I20260812 21:06:15.695674 3048 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 21:06:15.696352 3066 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:06:15.696521 3048 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 21:06:15.696609 3048 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786568607354921-12282-0/minicluster-data/master-0/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786568607354921-12282-0/minicluster-data/master-0/wal
uuid: "fef72df91dce4bd7a117a04d0ee76176"
format_stamp: "Formatted at 2026-08-12 21:06:15 on dist-test-slave-vpvm"
server_key: "2dafaf8f26615ae14d755d00ee18a140"
server_key_iv: "72cfcf167f10ee4fa2e98ba82c9dab5d"
server_key_version: "encryptionkey@0"
I20260812 21:06:15.696671 3048 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786568607354921-12282-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786568607354921-12282-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786568607354921-12282-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:06:15.716061 3048 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:06:15.718372 3048 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:06:15.718534 3048 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:06:15.722621 3048 rpc_server.cc:307] RPC server started. Bound to: 127.11.254.190:40331
I20260812 21:06:15.722678 3162 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.254.190:40331 every 8 connection(s)
I20260812 21:06:15.723063 3048 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
I20260812 21:06:15.724377 3163 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:06:15.726890 3163 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fef72df91dce4bd7a117a04d0ee76176: Bootstrap starting.
I20260812 21:06:15.727495 3163 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fef72df91dce4bd7a117a04d0ee76176: Neither blocks nor log segments found. Creating new log.
I20260812 21:06:15.727768 3163 log.cc:824] T 00000000000000000000000000000000 P fef72df91dce4bd7a117a04d0ee76176: Log is configured to *not* fsync() on all Append() calls
I20260812 21:06:15.728581 3163 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fef72df91dce4bd7a117a04d0ee76176: No bootstrap required, opened a new log
I20260812 21:06:15.730293 3163 raft_consensus.cc:348] T 00000000000000000000000000000000 P fef72df91dce4bd7a117a04d0ee76176 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fef72df91dce4bd7a117a04d0ee76176" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 40331 } }
I20260812 21:06:15.730443 3163 raft_consensus.cc:374] T 00000000000000000000000000000000 P fef72df91dce4bd7a117a04d0ee76176 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:06:15.730472 3163 raft_consensus.cc:729] T 00000000000000000000000000000000 P fef72df91dce4bd7a117a04d0ee76176 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fef72df91dce4bd7a117a04d0ee76176, State: Initialized, Role: FOLLOWER
I20260812 21:06:15.730624 3163 consensus_queue.cc:260] T 00000000000000000000000000000000 P fef72df91dce4bd7a117a04d0ee76176 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fef72df91dce4bd7a117a04d0ee76176" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 40331 } }
I20260812 21:06:15.730688 3163 raft_consensus.cc:388] T 00000000000000000000000000000000 P fef72df91dce4bd7a117a04d0ee76176 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 21:06:15.730710 3163 raft_consensus.cc:482] T 00000000000000000000000000000000 P fef72df91dce4bd7a117a04d0ee76176 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 21:06:15.730757 3163 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fef72df91dce4bd7a117a04d0ee76176 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:06:15.731297 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 3048
I20260812 21:06:15.731325 3163 raft_consensus.cc:504] T 00000000000000000000000000000000 P fef72df91dce4bd7a117a04d0ee76176 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fef72df91dce4bd7a117a04d0ee76176" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 40331 } }
I20260812 21:06:15.731393 12282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786568607354921-12282-0/minicluster-data/master-0/wal/instance
I20260812 21:06:15.731423 3163 leader_election.cc:302] T 00000000000000000000000000000000 P fef72df91dce4bd7a117a04d0ee76176 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fef72df91dce4bd7a117a04d0ee76176; no voters:
I20260812 21:06:15.731618 12282 external_mini_cluster.cc:1428] Setting key 078585a50c4b70cb675f772ac4328b6a
I20260812 21:06:15.731611 3163 leader_election.cc:288] T 00000000000000000000000000000000 P fef72df91dce4bd7a117a04d0ee76176 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 21:06:15.731647 3166 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fef72df91dce4bd7a117a04d0ee76176 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:06:15.731766 3166 raft_consensus.cc:686] T 00000000000000000000000000000000 P fef72df91dce4bd7a117a04d0ee76176 [term 1 LEADER]: Becoming Leader. State: Replica: fef72df91dce4bd7a117a04d0ee76176, State: Running, Role: LEADER
I20260812 21:06:15.731925 3163 sys_catalog.cc:564] T 00000000000000000000000000000000 P fef72df91dce4bd7a117a04d0ee76176 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 21:06:15.731917 3166 consensus_queue.cc:237] T 00000000000000000000000000000000 P fef72df91dce4bd7a117a04d0ee76176 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fef72df91dce4bd7a117a04d0ee76176" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 40331 } }
I20260812 21:06:15.732769 3168 sys_catalog.cc:455] T 00000000000000000000000000000000 P fef72df91dce4bd7a117a04d0ee76176 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fef72df91dce4bd7a117a04d0ee76176. Latest consensus state: current_term: 1 leader_uuid: "fef72df91dce4bd7a117a04d0ee76176" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fef72df91dce4bd7a117a04d0ee76176" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 40331 } } }
I20260812 21:06:15.732857 3168 sys_catalog.cc:458] T 00000000000000000000000000000000 P fef72df91dce4bd7a117a04d0ee76176 [sys.catalog]: This master's current role is: LEADER
I20260812 21:06:15.733214 3167 sys_catalog.cc:455] T 00000000000000000000000000000000 P fef72df91dce4bd7a117a04d0ee76176 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fef72df91dce4bd7a117a04d0ee76176" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fef72df91dce4bd7a117a04d0ee76176" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 40331 } } }
I20260812 21:06:15.733604 3183 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 21:06:15.734187 3167 sys_catalog.cc:458] T 00000000000000000000000000000000 P fef72df91dce4bd7a117a04d0ee76176 [sys.catalog]: This master's current role is: LEADER
I20260812 21:06:15.734625 3183 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 21:06:15.736227 3183 catalog_manager.cc:1347] Generated new cluster ID: 0a948b0f35dc4311a56fa99d9343fc05
I20260812 21:06:15.736284 3183 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 21:06:15 dist-test-slave-vpvm krb5kdc[3022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568775, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.254.190@KRBTEST.COM
I20260812 21:06:15.741925 3170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:15.732118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46236 (local address 127.11.254.190:40331)
0812 21:06:15.732302 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0812 21:06:15.732313 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 21:06:15.732343 (+ 30us) server_negotiation.cc:407] Connection header received
0812 21:06:15.732449 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:06:15.732455 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:06:15.732515 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:06:15.733264 (+ 749us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:06:15.733698 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:06:15.735047 (+ 1349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:06:15.736019 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:06:15.736548 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:06:15.736648 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:06:15.738879 (+ 2231us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:06:15.738883 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:06:15.738883 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:06:15.738896 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:06:15.740436 (+ 1540us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:06:15.740753 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:06:15.740757 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:06:15.740757 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:06:15.740851 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:06:15.741028 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:06:15.741029 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:06:15.741029 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:06:15.741356 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:06:15.741516 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0812 21:06:15.741626 (+ 110us) server_negotiation.cc:299] Negotiation successful
0812 21:06:15.741766 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":75,"threads_started":1}
I20260812 21:06:15.742913 12282 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 21:06:15.778409 3183 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 21:06:15.778995 3183 catalog_manager.cc:1504] Loading token signing keys...
I20260812 21:06:15.788040 3183 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fef72df91dce4bd7a117a04d0ee76176: Generated new TSK 0
I20260812 21:06:15.788276 3183 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 21:06:15.796648 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 3048
2026-08-12T21:06:15Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1527 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 21:06:15 dist-test-slave-vpvm krb5kdc[3199](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 21:06:15 dist-test-slave-vpvm krb5kdc[3199](info): set up 2 sockets
Aug 12 21:06:15 dist-test-slave-vpvm krb5kdc[3199](info): commencing operation
krb5kdc: starting...
W20260812 21:06:16.996451 12282 mini_kdc.cc:121] Time spent starting KDC: real 1.168s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/krb5kdc/test-user.keytab.
Aug 12 21:06:17 dist-test-slave-vpvm krb5kdc[3199](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568777, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T21:06:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T21:06:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.254.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.254.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.254.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.190.keytab.
Entry for principal kudu/127.11.254.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.190.keytab.
I20260812 21:06:17.125674 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.254.190:39351
--webserver_interface=127.11.254.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.190.keytab
--principal=kudu/127.11.254.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.254.148:41385
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.254.190:39351
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:06:17.211722 3220 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:06:17.211971 3220 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:06:17.212006 3220 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:06:17.213207 3220 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 21:06:17.213253 3220 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:06:17.213271 3220 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:06:17.213287 3220 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 21:06:17.213301 3220 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 21:06:17.214898 3220 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.254.148:41385
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.254.190:39351
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.190.keytab
--principal=kudu/127.11.254.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.254.190:39351
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.254.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3220
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:06:17.215178 3220 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:06:17.215404 3220 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:06:17.220497 3230 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:06:17.220602 3232 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:06:17.220646 3225 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:06:17.220726 3220 server_base.cc:1034] running on GCE node
I20260812 21:06:17.220981 3220 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:06:17.221329 3220 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:06:17.222484 3220 hybrid_clock.cc:648] HybridClock initialized: now 1786568777222447 us; error 48 us; skew 500 ppm
Aug 12 21:06:17 dist-test-slave-vpvm krb5kdc[3199](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568777, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:06:17.233255 3220 init.cc:377] Logged in from keytab as kudu/127.11.254.190@KRBTEST.COM (short username kudu)
I20260812 21:06:17.233822 3220 webserver.cc:460] Webserver started at http://127.11.254.190:41491/ using document root <none> and password file <none>
I20260812 21:06:17.234086 3220 fs_manager.cc:359] Metadata directory not provided
I20260812 21:06:17.234148 3220 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:06:17.234292 3220 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:06:17.235421 3220 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/master-0/data/instance:
uuid: "95eb7458d397406299827f3ca14509ae"
format_stamp: "Formatted at 2026-08-12 21:06:17 on dist-test-slave-vpvm"
server_key: "cf81d91cdcb203811592b175a7bc3fc7"
server_key_iv: "a1bef8cd145b42ee2459799f78eca1da"
server_key_version: "encryptionkey@0"
I20260812 21:06:17.235787 3220 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/master-0/wal/instance:
uuid: "95eb7458d397406299827f3ca14509ae"
format_stamp: "Formatted at 2026-08-12 21:06:17 on dist-test-slave-vpvm"
server_key: "cf81d91cdcb203811592b175a7bc3fc7"
server_key_iv: "a1bef8cd145b42ee2459799f78eca1da"
server_key_version: "encryptionkey@0"
I20260812 21:06:17.237180 3220 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 21:06:17.237927 3248 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:06:17.238088 3220 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 21:06:17.238195 3220 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/master-0/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/master-0/wal
uuid: "95eb7458d397406299827f3ca14509ae"
format_stamp: "Formatted at 2026-08-12 21:06:17 on dist-test-slave-vpvm"
server_key: "cf81d91cdcb203811592b175a7bc3fc7"
server_key_iv: "a1bef8cd145b42ee2459799f78eca1da"
server_key_version: "encryptionkey@0"
I20260812 21:06:17.238268 3220 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:06:17.252997 3220 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:06:17.255232 3220 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:06:17.255388 3220 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:06:17.259703 3220 rpc_server.cc:307] RPC server started. Bound to: 127.11.254.190:39351
I20260812 21:06:17.259742 3334 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.254.190:39351 every 8 connection(s)
I20260812 21:06:17.260092 3220 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
I20260812 21:06:17.261121 3335 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:06:17.261826 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 3220
I20260812 21:06:17.261924 12282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/master-0/wal/instance
I20260812 21:06:17.262135 12282 external_mini_cluster.cc:1428] Setting key e5abf336f69829ab3fb89b5f8d9615ed
I20260812 21:06:17.264335 3335 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 95eb7458d397406299827f3ca14509ae: Bootstrap starting.
I20260812 21:06:17.264943 3335 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 95eb7458d397406299827f3ca14509ae: Neither blocks nor log segments found. Creating new log.
I20260812 21:06:17.265239 3335 log.cc:824] T 00000000000000000000000000000000 P 95eb7458d397406299827f3ca14509ae: Log is configured to *not* fsync() on all Append() calls
I20260812 21:06:17.266039 3335 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 95eb7458d397406299827f3ca14509ae: No bootstrap required, opened a new log
I20260812 21:06:17.268309 3335 raft_consensus.cc:348] T 00000000000000000000000000000000 P 95eb7458d397406299827f3ca14509ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95eb7458d397406299827f3ca14509ae" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 39351 } }
I20260812 21:06:17.268507 3335 raft_consensus.cc:374] T 00000000000000000000000000000000 P 95eb7458d397406299827f3ca14509ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:06:17.268551 3335 raft_consensus.cc:729] T 00000000000000000000000000000000 P 95eb7458d397406299827f3ca14509ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95eb7458d397406299827f3ca14509ae, State: Initialized, Role: FOLLOWER
I20260812 21:06:17.268698 3335 consensus_queue.cc:260] T 00000000000000000000000000000000 P 95eb7458d397406299827f3ca14509ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95eb7458d397406299827f3ca14509ae" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 39351 } }
I20260812 21:06:17.268805 3335 raft_consensus.cc:388] T 00000000000000000000000000000000 P 95eb7458d397406299827f3ca14509ae [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 21:06:17.268832 3335 raft_consensus.cc:482] T 00000000000000000000000000000000 P 95eb7458d397406299827f3ca14509ae [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 21:06:17.268883 3335 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 95eb7458d397406299827f3ca14509ae [term 0 FOLLOWER]: Advancing to term 1
Aug 12 21:06:17 dist-test-slave-vpvm krb5kdc[3199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568777, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.254.190@KRBTEST.COM
I20260812 21:06:17.269510 3335 raft_consensus.cc:504] T 00000000000000000000000000000000 P 95eb7458d397406299827f3ca14509ae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95eb7458d397406299827f3ca14509ae" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 39351 } }
I20260812 21:06:17.269618 3335 leader_election.cc:302] T 00000000000000000000000000000000 P 95eb7458d397406299827f3ca14509ae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 95eb7458d397406299827f3ca14509ae; no voters:
I20260812 21:06:17.269797 3335 leader_election.cc:288] T 00000000000000000000000000000000 P 95eb7458d397406299827f3ca14509ae [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 21:06:17.269834 3342 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 95eb7458d397406299827f3ca14509ae [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:06:17.270006 3342 raft_consensus.cc:686] T 00000000000000000000000000000000 P 95eb7458d397406299827f3ca14509ae [term 1 LEADER]: Becoming Leader. State: Replica: 95eb7458d397406299827f3ca14509ae, State: Running, Role: LEADER
I20260812 21:06:17.270150 3335 sys_catalog.cc:564] T 00000000000000000000000000000000 P 95eb7458d397406299827f3ca14509ae [sys.catalog]: configured and running, proceeding with master startup.
I20260812 21:06:17.270172 3342 consensus_queue.cc:237] T 00000000000000000000000000000000 P 95eb7458d397406299827f3ca14509ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95eb7458d397406299827f3ca14509ae" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 39351 } }
I20260812 21:06:17.270529 3342 sys_catalog.cc:455] T 00000000000000000000000000000000 P 95eb7458d397406299827f3ca14509ae [sys.catalog]: SysCatalogTable state changed. Reason: New leader 95eb7458d397406299827f3ca14509ae. Latest consensus state: current_term: 1 leader_uuid: "95eb7458d397406299827f3ca14509ae" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95eb7458d397406299827f3ca14509ae" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 39351 } } }
I20260812 21:06:17.270578 3344 sys_catalog.cc:455] T 00000000000000000000000000000000 P 95eb7458d397406299827f3ca14509ae [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "95eb7458d397406299827f3ca14509ae" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95eb7458d397406299827f3ca14509ae" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 39351 } } }
I20260812 21:06:17.270669 3342 sys_catalog.cc:458] T 00000000000000000000000000000000 P 95eb7458d397406299827f3ca14509ae [sys.catalog]: This master's current role is: LEADER
I20260812 21:06:17.270675 3344 sys_catalog.cc:458] T 00000000000000000000000000000000 P 95eb7458d397406299827f3ca14509ae [sys.catalog]: This master's current role is: LEADER
I20260812 21:06:17.270972 3351 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 21:06:17.272116 3351 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 21:06:17.273633 3339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:17.262674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45978 (local address 127.11.254.190:39351)
0812 21:06:17.262834 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0812 21:06:17.262843 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 21:06:17.262869 (+ 26us) server_negotiation.cc:407] Connection header received
0812 21:06:17.263341 (+ 472us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:06:17.263349 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:06:17.263398 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:06:17.263514 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:06:17.264287 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:06:17.265274 (+ 987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:06:17.266351 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:06:17.267003 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:06:17.267095 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:06:17.269776 (+ 2681us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:06:17.269780 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:06:17.269781 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:06:17.269794 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:06:17.271924 (+ 2130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:06:17.272356 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:06:17.272360 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:06:17.272360 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:06:17.272470 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:06:17.272680 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:06:17.272681 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:06:17.272681 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:06:17.273005 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:06:17.273180 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0812 21:06:17.273280 (+ 100us) server_negotiation.cc:299] Negotiation successful
0812 21:06:17.273417 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":64,"threads_started":1}
I20260812 21:06:17.274173 3351 catalog_manager.cc:1347] Generated new cluster ID: 0ae449a4aa65496abf11b8a33ff708f6
I20260812 21:06:17.274220 3351 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 21:06:17.279815 3351 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 21:06:17.280339 3351 catalog_manager.cc:1504] Loading token signing keys...
I20260812 21:06:17.285266 3351 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 95eb7458d397406299827f3ca14509ae: Generated new TSK 0
I20260812 21:06:17.285458 3351 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.254.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.254.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.254.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.129.keytab.
Entry for principal kudu/127.11.254.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.129.keytab.
I20260812 21:06:17.316391 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.254.129:0
--local_ip_for_outbound_sockets=127.11.254.129
--webserver_interface=127.11.254.129
--webserver_port=0
--tserver_master_addrs=127.11.254.190:39351
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.129.keytab
--principal=kudu/127.11.254.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.254.148:41385
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:06:17.403987 3375 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:06:17.404194 3375 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:06:17.404227 3375 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:06:17.405433 3375 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:06:17.405480 3375 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:06:17.405515 3375 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.254.129
I20260812 21:06:17.407035 3375 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.254.148:41385
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.129.keytab
--principal=kudu/127.11.254.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.254.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.254.129
--webserver_port=0
--tserver_master_addrs=127.11.254.190:39351
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3375
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.254.129
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:06:17.407284 3375 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:06:17.407509 3375 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:06:17.412744 3388 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:06:17.412806 3382 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:06:17.412751 3381 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:06:17.412815 3375 server_base.cc:1034] running on GCE node
I20260812 21:06:17.413178 3375 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:06:17.413408 3375 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:06:17.414623 3375 hybrid_clock.cc:648] HybridClock initialized: now 1786568777414605 us; error 31 us; skew 500 ppm
Aug 12 21:06:17 dist-test-slave-vpvm krb5kdc[3199](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568777, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:06:17.417313 3375 init.cc:377] Logged in from keytab as kudu/127.11.254.129@KRBTEST.COM (short username kudu)
I20260812 21:06:17.417771 3375 webserver.cc:460] Webserver started at http://127.11.254.129:38073/ using document root <none> and password file <none>
I20260812 21:06:17.418036 3375 fs_manager.cc:359] Metadata directory not provided
I20260812 21:06:17.418088 3375 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:06:17.418192 3375 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:06:17.419265 3375 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/ts-0/data/instance:
uuid: "3ac4f761642144ab97f75d63be9843fd"
format_stamp: "Formatted at 2026-08-12 21:06:17 on dist-test-slave-vpvm"
server_key: "1f2072b5e42ba6bf4d89ffde48532cc7"
server_key_iv: "c1f4e2a6f8fbb9b8f6f50b7e63cda005"
server_key_version: "encryptionkey@0"
I20260812 21:06:17.419581 3375 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/ts-0/wal/instance:
uuid: "3ac4f761642144ab97f75d63be9843fd"
format_stamp: "Formatted at 2026-08-12 21:06:17 on dist-test-slave-vpvm"
server_key: "1f2072b5e42ba6bf4d89ffde48532cc7"
server_key_iv: "c1f4e2a6f8fbb9b8f6f50b7e63cda005"
server_key_version: "encryptionkey@0"
I20260812 21:06:17.420882 3375 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 21:06:17.421607 3401 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:06:17.421830 3375 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 21:06:17.421933 3375 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/ts-0/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/ts-0/wal
uuid: "3ac4f761642144ab97f75d63be9843fd"
format_stamp: "Formatted at 2026-08-12 21:06:17 on dist-test-slave-vpvm"
server_key: "1f2072b5e42ba6bf4d89ffde48532cc7"
server_key_iv: "c1f4e2a6f8fbb9b8f6f50b7e63cda005"
server_key_version: "encryptionkey@0"
I20260812 21:06:17.422052 3375 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:06:17.440092 3375 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:06:17.443030 3375 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:06:17.443188 3375 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:06:17.443452 3375 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:06:17.443823 3375 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:06:17.443859 3375 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:06:17.443884 3375 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:06:17.443912 3375 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:06:17.449842 3375 rpc_server.cc:307] RPC server started. Bound to: 127.11.254.129:35389
I20260812 21:06:17.449920 3589 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.254.129:35389 every 8 connection(s)
I20260812 21:06:17.450276 3375 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/ts-0/data/info.pb
I20260812 21:06:17.452963 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 3375
I20260812 21:06:17.453099 12282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/ts-0/wal/instance
I20260812 21:06:17.453317 12282 external_mini_cluster.cc:1428] Setting key 350a589fce018c9567a3d5f4627906ed
Aug 12 21:06:17 dist-test-slave-vpvm krb5kdc[3199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568777, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.129@KRBTEST.COM for kudu/127.11.254.190@KRBTEST.COM
I20260812 21:06:17.461562 3339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:17.451120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.129:53861 (local address 127.11.254.190:39351)
0812 21:06:17.451196 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0812 21:06:17.451198 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:06:17.451387 (+ 189us) server_negotiation.cc:407] Connection header received
0812 21:06:17.451923 (+ 536us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:06:17.451924 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:06:17.451934 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:06:17.452005 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:06:17.453326 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:06:17.454085 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:06:17.455562 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:06:17.455793 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:06:17.455867 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:06:17.458319 (+ 2452us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:06:17.458320 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:06:17.458321 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:06:17.458332 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:06:17.460034 (+ 1702us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:06:17.460475 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:06:17.460476 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:06:17.460476 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:06:17.460546 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:06:17.460788 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:06:17.460789 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:06:17.460789 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:06:17.460952 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:06:17.461025 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 21:06:17.461350 (+ 325us) server_negotiation.cc:299] Negotiation successful
0812 21:06:17.461457 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 21:06:17.461789 3592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:17.451065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:39351 (local address 127.11.254.129:53861)
0812 21:06:17.451254 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0812 21:06:17.451299 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 21:06:17.451822 (+ 523us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:06:17.452158 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:06:17.452162 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:06:17.452865 (+ 703us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:06:17.453182 (+ 317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:06:17.453190 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:06:17.454185 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:06:17.454187 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:06:17.455443 (+ 1256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:06:17.455447 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:06:17.455887 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:06:17.455889 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:06:17.456049 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:06:17.456602 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:06:17.456612 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:06:17.458197 (+ 1585us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:06:17.460135 (+ 1938us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:06:17.460145 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:06:17.460147 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:06:17.460397 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:06:17.460611 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:06:17.460612 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:06:17.460612 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:06:17.460731 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:06:17.461044 (+ 313us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:06:17.461052 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:06:17.461211 (+ 159us) client_negotiation.cc:769] Sending connection context
0812 21:06:17.461315 (+ 104us) client_negotiation.cc:240] Negotiation successful
0812 21:06:17.461531 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":103,"threads_started":1}
I20260812 21:06:17.462337 3590 heartbeater.cc:347] Connected to a master server at 127.11.254.190:39351
I20260812 21:06:17.462445 3590 heartbeater.cc:464] Registering TS with master...
I20260812 21:06:17.462659 3590 heartbeater.cc:511] Master 127.11.254.190:39351 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.11.254.130@KRBTEST.COM; defaulting to no policy
I20260812 21:06:17.463289 3278 ts_manager.cc:194] Registered new tserver with Master: 3ac4f761642144ab97f75d63be9843fd (127.11.254.129:35389)
I20260812 21:06:17.463939 3278 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.254.129@KRBTEST.COM'} at 127.11.254.129:53861
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.254.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.254.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.130.keytab.
Entry for principal kudu/127.11.254.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.130.keytab.
I20260812 21:06:17.482987 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.254.130:0
--local_ip_for_outbound_sockets=127.11.254.130
--webserver_interface=127.11.254.130
--webserver_port=0
--tserver_master_addrs=127.11.254.190:39351
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.130.keytab
--principal=kudu/127.11.254.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.254.148:41385
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:06:17.569134 3595 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:06:17.569357 3595 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:06:17.569383 3595 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:06:17.570695 3595 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:06:17.570748 3595 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:06:17.570778 3595 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.254.130
I20260812 21:06:17.572481 3595 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.254.148:41385
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.130.keytab
--principal=kudu/127.11.254.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.254.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.254.130
--webserver_port=0
--tserver_master_addrs=127.11.254.190:39351
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3595
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.254.130
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:06:17.572736 3595 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:06:17.572974 3595 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:06:17.578150 3602 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:06:17.578158 3605 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:06:17.578158 3603 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:06:17.578284 3595 server_base.cc:1034] running on GCE node
I20260812 21:06:17.578644 3595 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:06:17.578907 3595 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:06:17.580070 3595 hybrid_clock.cc:648] HybridClock initialized: now 1786568777580059 us; error 32 us; skew 500 ppm
Aug 12 21:06:17 dist-test-slave-vpvm krb5kdc[3199](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568777, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:06:17.582644 3595 init.cc:377] Logged in from keytab as kudu/127.11.254.130@KRBTEST.COM (short username kudu)
I20260812 21:06:17.583139 3595 webserver.cc:460] Webserver started at http://127.11.254.130:36717/ using document root <none> and password file <none>
I20260812 21:06:17.583374 3595 fs_manager.cc:359] Metadata directory not provided
I20260812 21:06:17.583420 3595 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:06:17.583551 3595 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:06:17.584613 3595 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/ts-1/data/instance:
uuid: "5fb1ffccc43e4fd9b9f078c167a38d55"
format_stamp: "Formatted at 2026-08-12 21:06:17 on dist-test-slave-vpvm"
server_key: "2675fb8c3ba201c19c227f1b3b8df4f8"
server_key_iv: "e7f4afd28c563ca1cdba616336e9feef"
server_key_version: "encryptionkey@0"
I20260812 21:06:17.584959 3595 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/ts-1/wal/instance:
uuid: "5fb1ffccc43e4fd9b9f078c167a38d55"
format_stamp: "Formatted at 2026-08-12 21:06:17 on dist-test-slave-vpvm"
server_key: "2675fb8c3ba201c19c227f1b3b8df4f8"
server_key_iv: "e7f4afd28c563ca1cdba616336e9feef"
server_key_version: "encryptionkey@0"
I20260812 21:06:17.586248 3595 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260812 21:06:17.587023 3617 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:06:17.587244 3595 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 21:06:17.587327 3595 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/ts-1/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/ts-1/wal
uuid: "5fb1ffccc43e4fd9b9f078c167a38d55"
format_stamp: "Formatted at 2026-08-12 21:06:17 on dist-test-slave-vpvm"
server_key: "2675fb8c3ba201c19c227f1b3b8df4f8"
server_key_iv: "e7f4afd28c563ca1cdba616336e9feef"
server_key_version: "encryptionkey@0"
I20260812 21:06:17.587393 3595 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:06:17.603906 3595 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:06:17.606278 3595 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:06:17.606429 3595 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:06:17.606685 3595 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:06:17.606993 3595 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:06:17.607029 3595 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:06:17.607059 3595 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:06:17.607080 3595 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:06:17.613098 3595 rpc_server.cc:307] RPC server started. Bound to: 127.11.254.130:46075
I20260812 21:06:17.613162 3810 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.254.130:46075 every 8 connection(s)
I20260812 21:06:17.613481 3595 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/ts-1/data/info.pb
I20260812 21:06:17.619283 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 3595
I20260812 21:06:17.619381 12282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/ts-1/wal/instance
I20260812 21:06:17.619557 12282 external_mini_cluster.cc:1428] Setting key 0c5fd1a611882bebb608553111a7ded2
Aug 12 21:06:17 dist-test-slave-vpvm krb5kdc[3199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568777, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.130@KRBTEST.COM for kudu/127.11.254.190@KRBTEST.COM
I20260812 21:06:17.623229 3339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:17.614345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.130:34861 (local address 127.11.254.190:39351)
0812 21:06:17.614399 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0812 21:06:17.614401 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:06:17.614579 (+ 178us) server_negotiation.cc:407] Connection header received
0812 21:06:17.615102 (+ 523us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:06:17.615103 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:06:17.615115 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:06:17.615174 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:06:17.616120 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:06:17.616664 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:06:17.617679 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:06:17.617888 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:06:17.617989 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:06:17.620243 (+ 2254us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:06:17.620244 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:06:17.620244 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:06:17.620250 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:06:17.621654 (+ 1404us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:06:17.622130 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:06:17.622131 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:06:17.622132 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:06:17.622203 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:06:17.622571 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:06:17.622571 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:06:17.622572 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:06:17.622692 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:06:17.622747 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0812 21:06:17.623084 (+ 337us) server_negotiation.cc:299] Negotiation successful
0812 21:06:17.623163 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260812 21:06:17.623521 3818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:17.614301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:39351 (local address 127.11.254.130:34861)
0812 21:06:17.614456 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0812 21:06:17.614501 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 21:06:17.614995 (+ 494us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:06:17.615238 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:06:17.615241 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:06:17.615821 (+ 580us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:06:17.616024 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:06:17.616029 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:06:17.616742 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:06:17.616743 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:06:17.617558 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:06:17.617560 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:06:17.617992 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:06:17.617993 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:06:17.618111 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:06:17.618655 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:06:17.618664 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:06:17.620156 (+ 1492us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:06:17.621743 (+ 1587us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:06:17.621751 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:06:17.621752 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:06:17.622033 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:06:17.622281 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:06:17.622282 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:06:17.622282 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:06:17.622447 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:06:17.622753 (+ 306us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:06:17.622760 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:06:17.622947 (+ 187us) client_negotiation.cc:769] Sending connection context
0812 21:06:17.623065 (+ 118us) client_negotiation.cc:240] Negotiation successful
0812 21:06:17.623266 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":79,"threads_started":1}
I20260812 21:06:17.624022 3811 heartbeater.cc:347] Connected to a master server at 127.11.254.190:39351
I20260812 21:06:17.624125 3811 heartbeater.cc:464] Registering TS with master...
I20260812 21:06:17.624312 3811 heartbeater.cc:511] Master 127.11.254.190:39351 requested a full tablet report, sending...
I20260812 21:06:17.624727 3278 ts_manager.cc:194] Registered new tserver with Master: 5fb1ffccc43e4fd9b9f078c167a38d55 (127.11.254.130:46075)
I20260812 21:06:17.625257 3278 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.254.130@KRBTEST.COM'} at 127.11.254.130:34861
WARNING: no policy specified for kudu/127.11.254.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.254.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.254.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.131.keytab.
Entry for principal kudu/127.11.254.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.131.keytab.
I20260812 21:06:17.647043 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.254.131:0
--local_ip_for_outbound_sockets=127.11.254.131
--webserver_interface=127.11.254.131
--webserver_port=0
--tserver_master_addrs=127.11.254.190:39351
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.131.keytab
--principal=kudu/127.11.254.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.254.148:41385
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:06:17.732939 3821 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:06:17.733165 3821 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:06:17.733192 3821 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:06:17.734418 3821 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:06:17.734467 3821 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:06:17.734493 3821 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.254.131
I20260812 21:06:17.735984 3821 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.254.148:41385
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.131.keytab
--principal=kudu/127.11.254.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.254.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.254.131
--webserver_port=0
--tserver_master_addrs=127.11.254.190:39351
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3821
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.254.131
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:06:17.736181 3821 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:06:17.736382 3821 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:06:17.741792 3829 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:06:17.741819 3828 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:06:17.741858 3821 server_base.cc:1034] running on GCE node
W20260812 21:06:17.741986 3831 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:06:17.742280 3821 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:06:17.742547 3821 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:06:17.743716 3821 hybrid_clock.cc:648] HybridClock initialized: now 1786568777743700 us; error 33 us; skew 500 ppm
Aug 12 21:06:17 dist-test-slave-vpvm krb5kdc[3199](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568777, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:06:17.746472 3821 init.cc:377] Logged in from keytab as kudu/127.11.254.131@KRBTEST.COM (short username kudu)
I20260812 21:06:17.747018 3821 webserver.cc:460] Webserver started at http://127.11.254.131:40205/ using document root <none> and password file <none>
I20260812 21:06:17.747231 3821 fs_manager.cc:359] Metadata directory not provided
I20260812 21:06:17.747278 3821 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:06:17.747407 3821 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:06:17.748509 3821 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/ts-2/data/instance:
uuid: "bbe9664da0ab4822b18a15afc541189b"
format_stamp: "Formatted at 2026-08-12 21:06:17 on dist-test-slave-vpvm"
server_key: "fef752db8fedde413b2ee07f590d945c"
server_key_iv: "aa8415e4b63fc4e866c2a11d014c8df3"
server_key_version: "encryptionkey@0"
I20260812 21:06:17.748878 3821 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/ts-2/wal/instance:
uuid: "bbe9664da0ab4822b18a15afc541189b"
format_stamp: "Formatted at 2026-08-12 21:06:17 on dist-test-slave-vpvm"
server_key: "fef752db8fedde413b2ee07f590d945c"
server_key_iv: "aa8415e4b63fc4e866c2a11d014c8df3"
server_key_version: "encryptionkey@0"
I20260812 21:06:17.750243 3821 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 21:06:17.751016 3843 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:06:17.751164 3821 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 21:06:17.751250 3821 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/ts-2/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/ts-2/wal
uuid: "bbe9664da0ab4822b18a15afc541189b"
format_stamp: "Formatted at 2026-08-12 21:06:17 on dist-test-slave-vpvm"
server_key: "fef752db8fedde413b2ee07f590d945c"
server_key_iv: "aa8415e4b63fc4e866c2a11d014c8df3"
server_key_version: "encryptionkey@0"
I20260812 21:06:17.751315 3821 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:06:17.768919 3821 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:06:17.771461 3821 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:06:17.771631 3821 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:06:17.771865 3821 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:06:17.772210 3821 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:06:17.772248 3821 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:06:17.772280 3821 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:06:17.772300 3821 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:06:17.778281 3821 rpc_server.cc:307] RPC server started. Bound to: 127.11.254.131:33971
I20260812 21:06:17.778347 4031 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.254.131:33971 every 8 connection(s)
I20260812 21:06:17.778676 3821 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/ts-2/data/info.pb
I20260812 21:06:17.782872 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 3821
I20260812 21:06:17.782986 12282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/ts-2/wal/instance
I20260812 21:06:17.783170 12282 external_mini_cluster.cc:1428] Setting key d4dd78f1a5c7f46b1104ca557327be76
Aug 12 21:06:17 dist-test-slave-vpvm krb5kdc[3199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568777, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.131@KRBTEST.COM for kudu/127.11.254.190@KRBTEST.COM
I20260812 21:06:17.788281 3339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:17.779412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.131:53079 (local address 127.11.254.190:39351)
0812 21:06:17.779505 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0812 21:06:17.779507 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:06:17.779660 (+ 153us) server_negotiation.cc:407] Connection header received
0812 21:06:17.780182 (+ 522us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:06:17.780183 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:06:17.780194 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:06:17.780242 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:06:17.781239 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:06:17.781819 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:06:17.782871 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:06:17.783197 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:06:17.783273 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:06:17.785652 (+ 2379us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:06:17.785654 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:06:17.785654 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:06:17.785662 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:06:17.786997 (+ 1335us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:06:17.787371 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:06:17.787373 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:06:17.787373 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:06:17.787432 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:06:17.787659 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:06:17.787660 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:06:17.787660 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:06:17.787777 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:06:17.787833 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0812 21:06:17.788108 (+ 275us) server_negotiation.cc:299] Negotiation successful
0812 21:06:17.788192 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 21:06:17.788463 4036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:17.779383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:39351 (local address 127.11.254.131:53079)
0812 21:06:17.779539 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0812 21:06:17.779578 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 21:06:17.780088 (+ 510us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:06:17.780303 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:06:17.780306 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:06:17.780910 (+ 604us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:06:17.781127 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:06:17.781132 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:06:17.781900 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:06:17.781900 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:06:17.782774 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:06:17.782778 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:06:17.783286 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:06:17.783287 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:06:17.783465 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:06:17.784050 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:06:17.784061 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:06:17.785564 (+ 1503us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:06:17.787075 (+ 1511us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:06:17.787082 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:06:17.787085 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:06:17.787306 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:06:17.787492 (+ 186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:06:17.787493 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:06:17.787493 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:06:17.787604 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:06:17.787831 (+ 227us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:06:17.787838 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:06:17.787987 (+ 149us) client_negotiation.cc:769] Sending connection context
0812 21:06:17.788078 (+ 91us) client_negotiation.cc:240] Negotiation successful
0812 21:06:17.788241 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":78,"threads_started":1}
I20260812 21:06:17.788885 4032 heartbeater.cc:347] Connected to a master server at 127.11.254.190:39351
I20260812 21:06:17.788986 4032 heartbeater.cc:464] Registering TS with master...
I20260812 21:06:17.789177 4032 heartbeater.cc:511] Master 127.11.254.190:39351 requested a full tablet report, sending...
I20260812 21:06:17.789548 3278 ts_manager.cc:194] Registered new tserver with Master: bbe9664da0ab4822b18a15afc541189b (127.11.254.131:33971)
I20260812 21:06:17.790009 3278 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.254.131@KRBTEST.COM'} at 127.11.254.131:53079
I20260812 21:06:17.794768 12282 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 21:06:17.802210 3339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:17.796062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45986 (local address 127.11.254.190:39351)
0812 21:06:17.796123 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0812 21:06:17.796124 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 21:06:17.796248 (+ 124us) server_negotiation.cc:407] Connection header received
0812 21:06:17.796284 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:06:17.796285 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:06:17.796294 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:06:17.796344 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:06:17.796903 (+ 559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:06:17.797398 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:06:17.798246 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:06:17.798475 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:06:17.798531 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:06:17.799768 (+ 1237us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:06:17.799769 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:06:17.799770 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:06:17.799776 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:06:17.801041 (+ 1265us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:06:17.801369 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:06:17.801369 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:06:17.801370 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:06:17.801462 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:06:17.801636 (+ 174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:06:17.801636 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:06:17.801637 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:06:17.801780 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:06:17.801851 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0812 21:06:17.802073 (+ 222us) server_negotiation.cc:299] Negotiation successful
0812 21:06:17.802142 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260812 21:06:17.803973 3278 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45986:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 21:06:17.805847 3278 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 21:06:17.812388 4047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:17.808262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.130:46075 (local address 127.0.0.1:51626)
0812 21:06:17.808656 (+ 394us) negotiation.cc:107] Waiting for socket to connect
0812 21:06:17.808698 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 21:06:17.808767 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:06:17.809106 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:06:17.809108 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:06:17.809110 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:06:17.809162 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:06:17.809164 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:06:17.810313 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:06:17.810314 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:06:17.811316 (+ 1002us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:06:17.811318 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:06:17.812084 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:06:17.812085 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:06:17.812167 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:06:17.812182 (+ 15us) client_negotiation.cc:769] Sending connection context
0812 21:06:17.812253 (+ 71us) client_negotiation.cc:240] Negotiation successful
0812 21:06:17.812304 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":361,"thread_start_us":67,"threads_started":1}
I20260812 21:06:17.812400 4050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:17.808455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51626 (local address 127.11.254.130:46075)
0812 21:06:17.808945 (+ 490us) server_negotiation.cc:206] Beginning negotiation
0812 21:06:17.808948 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 21:06:17.808958 (+ 10us) server_negotiation.cc:407] Connection header received
0812 21:06:17.809015 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:06:17.809017 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:06:17.809032 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:06:17.809106 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:06:17.809420 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:06:17.810220 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:06:17.811391 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:06:17.812005 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:06:17.812080 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:06:17.812137 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0812 21:06:17.812272 (+ 135us) server_negotiation.cc:299] Negotiation successful
0812 21:06:17.812314 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":456,"thread_start_us":97,"threads_started":1}
I20260812 21:06:17.813026 4051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:17.808524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.131:33971 (local address 127.0.0.1:36926)
0812 21:06:17.809272 (+ 748us) negotiation.cc:107] Waiting for socket to connect
0812 21:06:17.809282 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 21:06:17.809333 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:06:17.809516 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:06:17.809517 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:06:17.809519 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:06:17.809562 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:06:17.809564 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:06:17.810811 (+ 1247us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:06:17.810812 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:06:17.811818 (+ 1006us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:06:17.811819 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:06:17.812763 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:06:17.812764 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:06:17.812827 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:06:17.812833 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 21:06:17.812880 (+ 47us) client_negotiation.cc:240] Negotiation successful
0812 21:06:17.812925 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":728,"thread_start_us":63,"threads_started":1}
I20260812 21:06:17.813028 4046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:17.808262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.129:35389 (local address 127.0.0.1:34872)
0812 21:06:17.808712 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0812 21:06:17.808722 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 21:06:17.808767 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:06:17.809018 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:06:17.809021 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:06:17.809024 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:06:17.809100 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:06:17.809102 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:06:17.810479 (+ 1377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:06:17.810480 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:06:17.811603 (+ 1123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:06:17.811606 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:06:17.812725 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:06:17.812726 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:06:17.812797 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:06:17.812802 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 21:06:17.812866 (+ 64us) client_negotiation.cc:240] Negotiation successful
0812 21:06:17.812910 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":440,"mutex_wait_us":2,"thread_start_us":76,"threads_started":1}
I20260812 21:06:17.813235 4049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:17.808378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34872 (local address 127.11.254.129:35389)
0812 21:06:17.808769 (+ 391us) server_negotiation.cc:206] Beginning negotiation
0812 21:06:17.808773 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 21:06:17.808786 (+ 13us) server_negotiation.cc:407] Connection header received
0812 21:06:17.808879 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:06:17.808881 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:06:17.808901 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:06:17.808993 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:06:17.809192 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:06:17.810353 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:06:17.811690 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:06:17.812400 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:06:17.812622 (+ 222us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:06:17.812689 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 21:06:17.812982 (+ 293us) server_negotiation.cc:299] Negotiation successful
0812 21:06:17.813030 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":79,"threads_started":1}
I20260812 21:06:17.813267 4052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:17.808615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36926 (local address 127.11.254.131:33971)
0812 21:06:17.808862 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0812 21:06:17.808866 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 21:06:17.809355 (+ 489us) server_negotiation.cc:407] Connection header received
0812 21:06:17.809404 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:06:17.809406 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:06:17.809422 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:06:17.809503 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:06:17.809621 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:06:17.810706 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:06:17.811893 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:06:17.812605 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:06:17.812671 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:06:17.812726 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0812 21:06:17.813146 (+ 420us) server_negotiation.cc:299] Negotiation successful
0812 21:06:17.813195 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":216,"thread_start_us":89,"threads_started":1}
I20260812 21:06:17.813477 3705 tablet_service.cc:1467] Processing CreateTablet for tablet 05c3b55e3da04169af9fd6b3a2ae4813 (DEFAULT_TABLE table=test-table [id=d8e06fb696f1440d83b7c25b311826d8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:06:17.813817 3705 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 05c3b55e3da04169af9fd6b3a2ae4813. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:06:17.814647 3477 tablet_service.cc:1467] Processing CreateTablet for tablet 05c3b55e3da04169af9fd6b3a2ae4813 (DEFAULT_TABLE table=test-table [id=d8e06fb696f1440d83b7c25b311826d8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:06:17.814675 3922 tablet_service.cc:1467] Processing CreateTablet for tablet 05c3b55e3da04169af9fd6b3a2ae4813 (DEFAULT_TABLE table=test-table [id=d8e06fb696f1440d83b7c25b311826d8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:06:17.814962 3477 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 05c3b55e3da04169af9fd6b3a2ae4813. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:06:17.815018 3922 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 05c3b55e3da04169af9fd6b3a2ae4813. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:06:17.817507 4055 tablet_bootstrap.cc:492] T 05c3b55e3da04169af9fd6b3a2ae4813 P 3ac4f761642144ab97f75d63be9843fd: Bootstrap starting.
I20260812 21:06:17.817619 4053 tablet_bootstrap.cc:492] T 05c3b55e3da04169af9fd6b3a2ae4813 P 5fb1ffccc43e4fd9b9f078c167a38d55: Bootstrap starting.
I20260812 21:06:17.818145 4055 tablet_bootstrap.cc:654] T 05c3b55e3da04169af9fd6b3a2ae4813 P 3ac4f761642144ab97f75d63be9843fd: Neither blocks nor log segments found. Creating new log.
I20260812 21:06:17.818295 4053 tablet_bootstrap.cc:654] T 05c3b55e3da04169af9fd6b3a2ae4813 P 5fb1ffccc43e4fd9b9f078c167a38d55: Neither blocks nor log segments found. Creating new log.
I20260812 21:06:17.818490 4056 tablet_bootstrap.cc:492] T 05c3b55e3da04169af9fd6b3a2ae4813 P bbe9664da0ab4822b18a15afc541189b: Bootstrap starting.
I20260812 21:06:17.818768 4055 log.cc:824] T 05c3b55e3da04169af9fd6b3a2ae4813 P 3ac4f761642144ab97f75d63be9843fd: Log is configured to *not* fsync() on all Append() calls
I20260812 21:06:17.818817 4053 log.cc:824] T 05c3b55e3da04169af9fd6b3a2ae4813 P 5fb1ffccc43e4fd9b9f078c167a38d55: Log is configured to *not* fsync() on all Append() calls
I20260812 21:06:17.819116 4056 tablet_bootstrap.cc:654] T 05c3b55e3da04169af9fd6b3a2ae4813 P bbe9664da0ab4822b18a15afc541189b: Neither blocks nor log segments found. Creating new log.
I20260812 21:06:17.819509 4053 tablet_bootstrap.cc:492] T 05c3b55e3da04169af9fd6b3a2ae4813 P 5fb1ffccc43e4fd9b9f078c167a38d55: No bootstrap required, opened a new log
I20260812 21:06:17.819564 4053 ts_tablet_manager.cc:1397] T 05c3b55e3da04169af9fd6b3a2ae4813 P 5fb1ffccc43e4fd9b9f078c167a38d55: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260812 21:06:17.819672 4056 log.cc:824] T 05c3b55e3da04169af9fd6b3a2ae4813 P bbe9664da0ab4822b18a15afc541189b: Log is configured to *not* fsync() on all Append() calls
I20260812 21:06:17.819705 4055 tablet_bootstrap.cc:492] T 05c3b55e3da04169af9fd6b3a2ae4813 P 3ac4f761642144ab97f75d63be9843fd: No bootstrap required, opened a new log
I20260812 21:06:17.819762 4055 ts_tablet_manager.cc:1397] T 05c3b55e3da04169af9fd6b3a2ae4813 P 3ac4f761642144ab97f75d63be9843fd: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 21:06:17.820490 4056 tablet_bootstrap.cc:492] T 05c3b55e3da04169af9fd6b3a2ae4813 P bbe9664da0ab4822b18a15afc541189b: No bootstrap required, opened a new log
I20260812 21:06:17.820562 4056 ts_tablet_manager.cc:1397] T 05c3b55e3da04169af9fd6b3a2ae4813 P bbe9664da0ab4822b18a15afc541189b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 21:06:17.821175 4053 raft_consensus.cc:348] T 05c3b55e3da04169af9fd6b3a2ae4813 P 5fb1ffccc43e4fd9b9f078c167a38d55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fb1ffccc43e4fd9b9f078c167a38d55" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 46075 } } peers { permanent_uuid: "3ac4f761642144ab97f75d63be9843fd" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 35389 } } peers { permanent_uuid: "bbe9664da0ab4822b18a15afc541189b" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 33971 } }
I20260812 21:06:17.821308 4053 raft_consensus.cc:374] T 05c3b55e3da04169af9fd6b3a2ae4813 P 5fb1ffccc43e4fd9b9f078c167a38d55 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:06:17.821341 4053 raft_consensus.cc:729] T 05c3b55e3da04169af9fd6b3a2ae4813 P 5fb1ffccc43e4fd9b9f078c167a38d55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5fb1ffccc43e4fd9b9f078c167a38d55, State: Initialized, Role: FOLLOWER
I20260812 21:06:17.821453 4053 consensus_queue.cc:260] T 05c3b55e3da04169af9fd6b3a2ae4813 P 5fb1ffccc43e4fd9b9f078c167a38d55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fb1ffccc43e4fd9b9f078c167a38d55" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 46075 } } peers { permanent_uuid: "3ac4f761642144ab97f75d63be9843fd" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 35389 } } peers { permanent_uuid: "bbe9664da0ab4822b18a15afc541189b" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 33971 } }
I20260812 21:06:17.821724 4053 ts_tablet_manager.cc:1428] T 05c3b55e3da04169af9fd6b3a2ae4813 P 5fb1ffccc43e4fd9b9f078c167a38d55: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260812 21:06:17.821883 3811 heartbeater.cc:503] Master 127.11.254.190:39351 was elected leader, sending a full tablet report...
I20260812 21:06:17.821805 4055 raft_consensus.cc:348] T 05c3b55e3da04169af9fd6b3a2ae4813 P 3ac4f761642144ab97f75d63be9843fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fb1ffccc43e4fd9b9f078c167a38d55" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 46075 } } peers { permanent_uuid: "3ac4f761642144ab97f75d63be9843fd" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 35389 } } peers { permanent_uuid: "bbe9664da0ab4822b18a15afc541189b" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 33971 } }
I20260812 21:06:17.822009 4055 raft_consensus.cc:374] T 05c3b55e3da04169af9fd6b3a2ae4813 P 3ac4f761642144ab97f75d63be9843fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:06:17.822055 4055 raft_consensus.cc:729] T 05c3b55e3da04169af9fd6b3a2ae4813 P 3ac4f761642144ab97f75d63be9843fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ac4f761642144ab97f75d63be9843fd, State: Initialized, Role: FOLLOWER
I20260812 21:06:17.822168 4055 consensus_queue.cc:260] T 05c3b55e3da04169af9fd6b3a2ae4813 P 3ac4f761642144ab97f75d63be9843fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fb1ffccc43e4fd9b9f078c167a38d55" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 46075 } } peers { permanent_uuid: "3ac4f761642144ab97f75d63be9843fd" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 35389 } } peers { permanent_uuid: "bbe9664da0ab4822b18a15afc541189b" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 33971 } }
I20260812 21:06:17.822434 4055 ts_tablet_manager.cc:1428] T 05c3b55e3da04169af9fd6b3a2ae4813 P 3ac4f761642144ab97f75d63be9843fd: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 21:06:17.822454 3590 heartbeater.cc:503] Master 127.11.254.190:39351 was elected leader, sending a full tablet report...
I20260812 21:06:17.822752 4056 raft_consensus.cc:348] T 05c3b55e3da04169af9fd6b3a2ae4813 P bbe9664da0ab4822b18a15afc541189b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fb1ffccc43e4fd9b9f078c167a38d55" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 46075 } } peers { permanent_uuid: "3ac4f761642144ab97f75d63be9843fd" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 35389 } } peers { permanent_uuid: "bbe9664da0ab4822b18a15afc541189b" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 33971 } }
I20260812 21:06:17.822882 4056 raft_consensus.cc:374] T 05c3b55e3da04169af9fd6b3a2ae4813 P bbe9664da0ab4822b18a15afc541189b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:06:17.822906 4056 raft_consensus.cc:729] T 05c3b55e3da04169af9fd6b3a2ae4813 P bbe9664da0ab4822b18a15afc541189b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bbe9664da0ab4822b18a15afc541189b, State: Initialized, Role: FOLLOWER
I20260812 21:06:17.823016 4056 consensus_queue.cc:260] T 05c3b55e3da04169af9fd6b3a2ae4813 P bbe9664da0ab4822b18a15afc541189b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fb1ffccc43e4fd9b9f078c167a38d55" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 46075 } } peers { permanent_uuid: "3ac4f761642144ab97f75d63be9843fd" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 35389 } } peers { permanent_uuid: "bbe9664da0ab4822b18a15afc541189b" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 33971 } }
I20260812 21:06:17.823295 4056 ts_tablet_manager.cc:1428] T 05c3b55e3da04169af9fd6b3a2ae4813 P bbe9664da0ab4822b18a15afc541189b: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 21:06:17.823307 4032 heartbeater.cc:503] Master 127.11.254.190:39351 was elected leader, sending a full tablet report...
W20260812 21:06:17.864169 3815 tablet.cc:2369] T 05c3b55e3da04169af9fd6b3a2ae4813 P 5fb1ffccc43e4fd9b9f078c167a38d55: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 21:06:17.951104 3591 tablet.cc:2369] T 05c3b55e3da04169af9fd6b3a2ae4813 P 3ac4f761642144ab97f75d63be9843fd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 21:06:17.979825 4065 raft_consensus.cc:482] T 05c3b55e3da04169af9fd6b3a2ae4813 P 5fb1ffccc43e4fd9b9f078c167a38d55 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 21:06:17.979997 4065 raft_consensus.cc:504] T 05c3b55e3da04169af9fd6b3a2ae4813 P 5fb1ffccc43e4fd9b9f078c167a38d55 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fb1ffccc43e4fd9b9f078c167a38d55" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 46075 } } peers { permanent_uuid: "3ac4f761642144ab97f75d63be9843fd" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 35389 } } peers { permanent_uuid: "bbe9664da0ab4822b18a15afc541189b" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 33971 } }
I20260812 21:06:17.980489 4065 leader_election.cc:288] T 05c3b55e3da04169af9fd6b3a2ae4813 P 5fb1ffccc43e4fd9b9f078c167a38d55 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ac4f761642144ab97f75d63be9843fd (127.11.254.129:35389), bbe9664da0ab4822b18a15afc541189b (127.11.254.131:33971)
I20260812 21:06:17.983417 4049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:17.980869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.130:37075 (local address 127.11.254.129:35389)
0812 21:06:17.980923 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0812 21:06:17.980926 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 21:06:17.980938 (+ 12us) server_negotiation.cc:407] Connection header received
0812 21:06:17.980971 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:06:17.980972 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:06:17.980984 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:06:17.981046 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:06:17.981211 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:06:17.981719 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:06:17.982721 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:06:17.983074 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:06:17.983147 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:06:17.983157 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0812 21:06:17.983307 (+ 150us) server_negotiation.cc:299] Negotiation successful
0812 21:06:17.983347 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260812 21:06:17.983448 3818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:17.980726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.129:35389 (local address 127.11.254.130:37075)
0812 21:06:17.980816 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0812 21:06:17.980831 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 21:06:17.980902 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:06:17.981065 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:06:17.981066 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:06:17.981068 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:06:17.981145 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:06:17.981147 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:06:17.981805 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:06:17.981806 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:06:17.982627 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:06:17.982630 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:06:17.983162 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:06:17.983163 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:06:17.983241 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:06:17.983245 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 21:06:17.983292 (+ 47us) client_negotiation.cc:240] Negotiation successful
0812 21:06:17.983336 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":64}
I20260812 21:06:17.983469 4052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:17.980941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.130:48173 (local address 127.11.254.131:33971)
0812 21:06:17.980995 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0812 21:06:17.980997 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:06:17.981094 (+ 97us) server_negotiation.cc:407] Connection header received
0812 21:06:17.981144 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:06:17.981145 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:06:17.981154 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:06:17.981199 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:06:17.981332 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:06:17.981813 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:06:17.982762 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:06:17.983157 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:06:17.983220 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:06:17.983229 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 21:06:17.983361 (+ 132us) server_negotiation.cc:299] Negotiation successful
0812 21:06:17.983399 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260812 21:06:17.983724 4069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:17.980854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.131:33971 (local address 127.11.254.130:48173)
0812 21:06:17.981028 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0812 21:06:17.981037 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 21:06:17.981091 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:06:17.981207 (+ 116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:06:17.981207 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:06:17.981209 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:06:17.981278 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:06:17.981280 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:06:17.981903 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:06:17.981904 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:06:17.982679 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:06:17.982682 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:06:17.983235 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:06:17.983236 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:06:17.983303 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:06:17.983306 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 21:06:17.983351 (+ 45us) client_negotiation.cc:240] Negotiation successful
0812 21:06:17.983390 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":111,"threads_started":1}
I20260812 21:06:17.983848 3520 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05c3b55e3da04169af9fd6b3a2ae4813" candidate_uuid: "5fb1ffccc43e4fd9b9f078c167a38d55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ac4f761642144ab97f75d63be9843fd" is_pre_election: true
I20260812 21:06:17.984032 3520 raft_consensus.cc:2454] T 05c3b55e3da04169af9fd6b3a2ae4813 P 3ac4f761642144ab97f75d63be9843fd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5fb1ffccc43e4fd9b9f078c167a38d55 in term 0.
I20260812 21:06:17.984081 3954 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05c3b55e3da04169af9fd6b3a2ae4813" candidate_uuid: "5fb1ffccc43e4fd9b9f078c167a38d55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bbe9664da0ab4822b18a15afc541189b" is_pre_election: true
I20260812 21:06:17.984263 3954 raft_consensus.cc:2454] T 05c3b55e3da04169af9fd6b3a2ae4813 P bbe9664da0ab4822b18a15afc541189b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5fb1ffccc43e4fd9b9f078c167a38d55 in term 0.
I20260812 21:06:17.984270 3618 leader_election.cc:302] T 05c3b55e3da04169af9fd6b3a2ae4813 P 5fb1ffccc43e4fd9b9f078c167a38d55 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ac4f761642144ab97f75d63be9843fd, 5fb1ffccc43e4fd9b9f078c167a38d55; no voters:
I20260812 21:06:17.984406 4065 raft_consensus.cc:2781] T 05c3b55e3da04169af9fd6b3a2ae4813 P 5fb1ffccc43e4fd9b9f078c167a38d55 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 21:06:17.984491 4065 raft_consensus.cc:482] T 05c3b55e3da04169af9fd6b3a2ae4813 P 5fb1ffccc43e4fd9b9f078c167a38d55 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 21:06:17.984553 4065 raft_consensus.cc:3037] T 05c3b55e3da04169af9fd6b3a2ae4813 P 5fb1ffccc43e4fd9b9f078c167a38d55 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:06:17.985281 4065 raft_consensus.cc:504] T 05c3b55e3da04169af9fd6b3a2ae4813 P 5fb1ffccc43e4fd9b9f078c167a38d55 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fb1ffccc43e4fd9b9f078c167a38d55" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 46075 } } peers { permanent_uuid: "3ac4f761642144ab97f75d63be9843fd" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 35389 } } peers { permanent_uuid: "bbe9664da0ab4822b18a15afc541189b" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 33971 } }
I20260812 21:06:17.985406 4065 leader_election.cc:288] T 05c3b55e3da04169af9fd6b3a2ae4813 P 5fb1ffccc43e4fd9b9f078c167a38d55 [CANDIDATE]: Term 1 election: Requested vote from peers 3ac4f761642144ab97f75d63be9843fd (127.11.254.129:35389), bbe9664da0ab4822b18a15afc541189b (127.11.254.131:33971)
I20260812 21:06:17.985589 3954 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05c3b55e3da04169af9fd6b3a2ae4813" candidate_uuid: "5fb1ffccc43e4fd9b9f078c167a38d55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bbe9664da0ab4822b18a15afc541189b"
I20260812 21:06:17.985601 3520 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05c3b55e3da04169af9fd6b3a2ae4813" candidate_uuid: "5fb1ffccc43e4fd9b9f078c167a38d55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ac4f761642144ab97f75d63be9843fd"
I20260812 21:06:17.985671 3520 raft_consensus.cc:3037] T 05c3b55e3da04169af9fd6b3a2ae4813 P 3ac4f761642144ab97f75d63be9843fd [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:06:17.985664 3954 raft_consensus.cc:3037] T 05c3b55e3da04169af9fd6b3a2ae4813 P bbe9664da0ab4822b18a15afc541189b [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:06:17.986440 3954 raft_consensus.cc:2454] T 05c3b55e3da04169af9fd6b3a2ae4813 P bbe9664da0ab4822b18a15afc541189b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5fb1ffccc43e4fd9b9f078c167a38d55 in term 1.
I20260812 21:06:17.986442 3520 raft_consensus.cc:2454] T 05c3b55e3da04169af9fd6b3a2ae4813 P 3ac4f761642144ab97f75d63be9843fd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5fb1ffccc43e4fd9b9f078c167a38d55 in term 1.
I20260812 21:06:17.986595 3618 leader_election.cc:302] T 05c3b55e3da04169af9fd6b3a2ae4813 P 5fb1ffccc43e4fd9b9f078c167a38d55 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5fb1ffccc43e4fd9b9f078c167a38d55, bbe9664da0ab4822b18a15afc541189b; no voters:
I20260812 21:06:17.986694 4065 raft_consensus.cc:2781] T 05c3b55e3da04169af9fd6b3a2ae4813 P 5fb1ffccc43e4fd9b9f078c167a38d55 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:06:17.986850 4065 raft_consensus.cc:686] T 05c3b55e3da04169af9fd6b3a2ae4813 P 5fb1ffccc43e4fd9b9f078c167a38d55 [term 1 LEADER]: Becoming Leader. State: Replica: 5fb1ffccc43e4fd9b9f078c167a38d55, State: Running, Role: LEADER
I20260812 21:06:17.986963 4065 consensus_queue.cc:237] T 05c3b55e3da04169af9fd6b3a2ae4813 P 5fb1ffccc43e4fd9b9f078c167a38d55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fb1ffccc43e4fd9b9f078c167a38d55" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 46075 } } peers { permanent_uuid: "3ac4f761642144ab97f75d63be9843fd" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 35389 } } peers { permanent_uuid: "bbe9664da0ab4822b18a15afc541189b" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 33971 } }
I20260812 21:06:17.987763 3277 catalog_manager.cc:5495] T 05c3b55e3da04169af9fd6b3a2ae4813 P 5fb1ffccc43e4fd9b9f078c167a38d55 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5fb1ffccc43e4fd9b9f078c167a38d55 (127.11.254.130). New cstate: current_term: 1 leader_uuid: "5fb1ffccc43e4fd9b9f078c167a38d55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fb1ffccc43e4fd9b9f078c167a38d55" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 46075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ac4f761642144ab97f75d63be9843fd" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 35389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bbe9664da0ab4822b18a15afc541189b" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 33971 } health_report { overall_health: UNKNOWN } } }
I20260812 21:06:17.997923 4050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:17.995783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51642 (local address 127.11.254.130:46075)
0812 21:06:17.995839 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0812 21:06:17.995841 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:06:17.995848 (+ 7us) server_negotiation.cc:407] Connection header received
0812 21:06:17.995879 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:06:17.995880 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:06:17.995897 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:06:17.995958 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 21:06:17.996067 (+ 109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:06:17.996579 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:06:17.997299 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:06:17.997501 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:06:17.997564 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:06:17.997655 (+ 91us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 21:06:17.997752 (+ 97us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 21:06:17.997818 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0812 21:06:17.997829 (+ 11us) server_negotiation.cc:299] Negotiation successful
0812 21:06:17.997847 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260812 21:06:17.999703 3954 raft_consensus.cc:1261] T 05c3b55e3da04169af9fd6b3a2ae4813 P bbe9664da0ab4822b18a15afc541189b [term 1 FOLLOWER]: Refusing update from remote peer 5fb1ffccc43e4fd9b9f078c167a38d55: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 21:06:17.999722 3520 raft_consensus.cc:1261] T 05c3b55e3da04169af9fd6b3a2ae4813 P 3ac4f761642144ab97f75d63be9843fd [term 1 FOLLOWER]: Refusing update from remote peer 5fb1ffccc43e4fd9b9f078c167a38d55: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 21:06:18.000028 4065 consensus_queue.cc:1035] T 05c3b55e3da04169af9fd6b3a2ae4813 P 5fb1ffccc43e4fd9b9f078c167a38d55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bbe9664da0ab4822b18a15afc541189b" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 33971 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 21:06:18.000166 4070 consensus_queue.cc:1035] T 05c3b55e3da04169af9fd6b3a2ae4813 P 5fb1ffccc43e4fd9b9f078c167a38d55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ac4f761642144ab97f75d63be9843fd" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 35389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 21:06:18.002821 4078 mvcc.cc:205] Tried to move back new op lower bound from 7317785714684375040 to 7317785714635173888. Current Snapshot: MvccSnapshot[applied={T|T < 7317785714684375040}]
I20260812 21:06:18.003192 4079 mvcc.cc:205] Tried to move back new op lower bound from 7317785714684375040 to 7317785714635173888. Current Snapshot: MvccSnapshot[applied={T|T < 7317785714684375040 or (T in {7317785714684375040})}]
I20260812 21:06:18.004837 3280 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45986:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 21:06:18.004923 3280 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45986:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 21:06:18.005618 3280 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 95eb7458d397406299827f3ca14509ae: Sending DeleteTablet for 3 replicas of tablet 05c3b55e3da04169af9fd6b3a2ae4813
I20260812 21:06:18.005970 3705 tablet_service.cc:1514] Processing DeleteTablet for tablet 05c3b55e3da04169af9fd6b3a2ae4813 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:06:17 UTC) from {username='kudu', principal='kudu/127.11.254.190@KRBTEST.COM'} at 127.0.0.1:51626
I20260812 21:06:18.006050 3477 tablet_service.cc:1514] Processing DeleteTablet for tablet 05c3b55e3da04169af9fd6b3a2ae4813 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:06:17 UTC) from {username='kudu', principal='kudu/127.11.254.190@KRBTEST.COM'} at 127.0.0.1:34872
I20260812 21:06:18.006291 3922 tablet_service.cc:1514] Processing DeleteTablet for tablet 05c3b55e3da04169af9fd6b3a2ae4813 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:06:17 UTC) from {username='kudu', principal='kudu/127.11.254.190@KRBTEST.COM'} at 127.0.0.1:36926
I20260812 21:06:18.006524 4094 tablet_replica.cc:330] T 05c3b55e3da04169af9fd6b3a2ae4813 P bbe9664da0ab4822b18a15afc541189b: stopping tablet replica
I20260812 21:06:18.006594 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 3375
I20260812 21:06:18.006667 4094 raft_consensus.cc:2229] T 05c3b55e3da04169af9fd6b3a2ae4813 P bbe9664da0ab4822b18a15afc541189b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 21:06:18.032555 4094 raft_consensus.cc:2258] T 05c3b55e3da04169af9fd6b3a2ae4813 P bbe9664da0ab4822b18a15afc541189b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 21:06:18.034160 4094 ts_tablet_manager.cc:1905] T 05c3b55e3da04169af9fd6b3a2ae4813 P bbe9664da0ab4822b18a15afc541189b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 21:06:18.035373 4089 tablet_replica.cc:330] T 05c3b55e3da04169af9fd6b3a2ae4813 P 5fb1ffccc43e4fd9b9f078c167a38d55: stopping tablet replica
I20260812 21:06:18.035602 4089 raft_consensus.cc:2229] T 05c3b55e3da04169af9fd6b3a2ae4813 P 5fb1ffccc43e4fd9b9f078c167a38d55 [term 1 LEADER]: Raft consensus shutting down.
I20260812 21:06:18.035861 4089 raft_consensus.cc:2258] T 05c3b55e3da04169af9fd6b3a2ae4813 P 5fb1ffccc43e4fd9b9f078c167a38d55 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 21:06:18.036382 4089 ts_tablet_manager.cc:1905] T 05c3b55e3da04169af9fd6b3a2ae4813 P 5fb1ffccc43e4fd9b9f078c167a38d55: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 21:06:18.036921 4094 ts_tablet_manager.cc:1918] T 05c3b55e3da04169af9fd6b3a2ae4813 P bbe9664da0ab4822b18a15afc541189b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 21:06:18.037020 4094 log.cc:1196] T 05c3b55e3da04169af9fd6b3a2ae4813 P bbe9664da0ab4822b18a15afc541189b: Deleting WAL directory at /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/ts-2/wal/wals/05c3b55e3da04169af9fd6b3a2ae4813
I20260812 21:06:18.037333 4094 ts_tablet_manager.cc:1939] T 05c3b55e3da04169af9fd6b3a2ae4813 P bbe9664da0ab4822b18a15afc541189b: Deleting consensus metadata
I20260812 21:06:18.037851 3250 catalog_manager.cc:4841] TS bbe9664da0ab4822b18a15afc541189b (127.11.254.131:33971): tablet 05c3b55e3da04169af9fd6b3a2ae4813 (table test-table [id=d8e06fb696f1440d83b7c25b311826d8]) successfully deleted
I20260812 21:06:18.038283 4089 ts_tablet_manager.cc:1918] T 05c3b55e3da04169af9fd6b3a2ae4813 P 5fb1ffccc43e4fd9b9f078c167a38d55: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 21:06:18.038354 4089 log.cc:1196] T 05c3b55e3da04169af9fd6b3a2ae4813 P 5fb1ffccc43e4fd9b9f078c167a38d55: Deleting WAL directory at /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/ts-1/wal/wals/05c3b55e3da04169af9fd6b3a2ae4813
I20260812 21:06:18.038578 4089 ts_tablet_manager.cc:1939] T 05c3b55e3da04169af9fd6b3a2ae4813 P 5fb1ffccc43e4fd9b9f078c167a38d55: Deleting consensus metadata
I20260812 21:06:18.038997 3253 catalog_manager.cc:4841] TS 5fb1ffccc43e4fd9b9f078c167a38d55 (127.11.254.130:46075): tablet 05c3b55e3da04169af9fd6b3a2ae4813 (table test-table [id=d8e06fb696f1440d83b7c25b311826d8]) successfully deleted
W20260812 21:06:18.042658 3250 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.254.129:35389 (error 108)
I20260812 21:06:18.042944 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 3595
I20260812 21:06:18.047794 4051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:18.043139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.129:35389 (local address 127.0.0.1:34876)
0812 21:06:18.047666 (+ 4527us) negotiation.cc:107] Waiting for socket to connect
0812 21:06:18.047744 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.129:35389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":4493}
W20260812 21:06:18.055523 3250 catalog_manager.cc:4584] TS 3ac4f761642144ab97f75d63be9843fd (127.11.254.129:35389): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 05c3b55e3da04169af9fd6b3a2ae4813: Network error: Client connection negotiation failed: client connection to 127.11.254.129:35389: connect: Connection refused (error 111)
I20260812 21:06:18.091132 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 3821
I20260812 21:06:18.108932 4051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:18.108746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.129:35389 (local address 127.0.0.1:34884)
0812 21:06:18.108829 (+ 83us) negotiation.cc:107] Waiting for socket to connect
0812 21:06:18.108873 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.129:35389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260812 21:06:18.109341 4051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:18.109248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.129:35389 (local address 127.0.0.1:34890)
0812 21:06:18.109288 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0812 21:06:18.109314 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.129:35389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260812 21:06:18.116179 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 3220
I20260812 21:06:18.134666 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.254.190:39351
--webserver_interface=127.11.254.190
--webserver_port=41491
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.190.keytab
--principal=kudu/127.11.254.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.254.148:41385
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.254.190:39351
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:06:18.218533 4099 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:06:18.218767 4099 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:06:18.218801 4099 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:06:18.219980 4099 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 21:06:18.220019 4099 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:06:18.220037 4099 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:06:18.220053 4099 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 21:06:18.220068 4099 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 21:06:18.221577 4099 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.254.148:41385
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.11.254.190:39351
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.190.keytab
--principal=kudu/127.11.254.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.254.190:39351
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.254.190
--webserver_port=41491
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4099
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:06:18.221837 4099 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:06:18.222077 4099 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:06:18.231442 4111 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:06:18.231556 4099 server_base.cc:1034] running on GCE node
W20260812 21:06:18.231412 4114 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:06:18.231412 4108 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:06:18.231930 4099 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:06:18.232266 4099 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:06:18.233415 4099 hybrid_clock.cc:648] HybridClock initialized: now 1786568778233397 us; error 38 us; skew 500 ppm
Aug 12 21:06:18 dist-test-slave-vpvm krb5kdc[3199](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568778, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:06:18.236014 4099 init.cc:377] Logged in from keytab as kudu/127.11.254.190@KRBTEST.COM (short username kudu)
I20260812 21:06:18.236454 4099 webserver.cc:460] Webserver started at http://127.11.254.190:41491/ using document root <none> and password file <none>
I20260812 21:06:18.236685 4099 fs_manager.cc:359] Metadata directory not provided
I20260812 21:06:18.236750 4099 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:06:18.238447 4099 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260812 21:06:18.239126 4125 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:06:18.239311 4099 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 21:06:18.239401 4099 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/master-0/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/master-0/wal
uuid: "95eb7458d397406299827f3ca14509ae"
format_stamp: "Formatted at 2026-08-12 21:06:17 on dist-test-slave-vpvm"
server_key: "cf81d91cdcb203811592b175a7bc3fc7"
server_key_iv: "a1bef8cd145b42ee2459799f78eca1da"
server_key_version: "encryptionkey@0"
I20260812 21:06:18.239816 4099 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:06:18.262619 4099 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:06:18.264932 4099 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:06:18.265082 4099 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:06:18.269409 4099 rpc_server.cc:307] RPC server started. Bound to: 127.11.254.190:39351
I20260812 21:06:18.269459 4207 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.254.190:39351 every 8 connection(s)
I20260812 21:06:18.269862 4099 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
I20260812 21:06:18.270418 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 4099
I20260812 21:06:18.270744 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.254.129:35389
--local_ip_for_outbound_sockets=127.11.254.129
--tserver_master_addrs=127.11.254.190:39351
--webserver_port=38073
--webserver_interface=127.11.254.129
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.129.keytab
--principal=kudu/127.11.254.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.254.148:41385
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 21:06:18.272179 4209 sys_catalog.cc:263] Verifying existing consensus state
I20260812 21:06:18.272964 4209 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 95eb7458d397406299827f3ca14509ae: Bootstrap starting.
I20260812 21:06:18.282259 4209 log.cc:824] T 00000000000000000000000000000000 P 95eb7458d397406299827f3ca14509ae: Log is configured to *not* fsync() on all Append() calls
I20260812 21:06:18.284586 4209 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 95eb7458d397406299827f3ca14509ae: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 21:06:18.284904 4209 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 95eb7458d397406299827f3ca14509ae: Bootstrap complete.
I20260812 21:06:18.287315 4209 raft_consensus.cc:348] T 00000000000000000000000000000000 P 95eb7458d397406299827f3ca14509ae [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95eb7458d397406299827f3ca14509ae" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 39351 } }
I20260812 21:06:18.287588 4209 raft_consensus.cc:729] T 00000000000000000000000000000000 P 95eb7458d397406299827f3ca14509ae [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95eb7458d397406299827f3ca14509ae, State: Initialized, Role: FOLLOWER
I20260812 21:06:18.287738 4209 consensus_queue.cc:260] T 00000000000000000000000000000000 P 95eb7458d397406299827f3ca14509ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95eb7458d397406299827f3ca14509ae" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 39351 } }
I20260812 21:06:18.287802 4209 raft_consensus.cc:388] T 00000000000000000000000000000000 P 95eb7458d397406299827f3ca14509ae [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 21:06:18.287848 4209 raft_consensus.cc:482] T 00000000000000000000000000000000 P 95eb7458d397406299827f3ca14509ae [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 21:06:18.287896 4209 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 95eb7458d397406299827f3ca14509ae [term 1 FOLLOWER]: Advancing to term 2
I20260812 21:06:18.288933 4209 raft_consensus.cc:504] T 00000000000000000000000000000000 P 95eb7458d397406299827f3ca14509ae [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95eb7458d397406299827f3ca14509ae" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 39351 } }
I20260812 21:06:18.289023 4209 leader_election.cc:302] T 00000000000000000000000000000000 P 95eb7458d397406299827f3ca14509ae [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 95eb7458d397406299827f3ca14509ae; no voters:
I20260812 21:06:18.289194 4209 leader_election.cc:288] T 00000000000000000000000000000000 P 95eb7458d397406299827f3ca14509ae [CANDIDATE]: Term 2 election: Requested vote from peers
I20260812 21:06:18.289248 4214 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 95eb7458d397406299827f3ca14509ae [term 2 FOLLOWER]: Leader election won for term 2
I20260812 21:06:18.289366 4214 raft_consensus.cc:686] T 00000000000000000000000000000000 P 95eb7458d397406299827f3ca14509ae [term 2 LEADER]: Becoming Leader. State: Replica: 95eb7458d397406299827f3ca14509ae, State: Running, Role: LEADER
I20260812 21:06:18.289453 4214 consensus_queue.cc:237] T 00000000000000000000000000000000 P 95eb7458d397406299827f3ca14509ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95eb7458d397406299827f3ca14509ae" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 39351 } }
I20260812 21:06:18.289484 4209 sys_catalog.cc:564] T 00000000000000000000000000000000 P 95eb7458d397406299827f3ca14509ae [sys.catalog]: configured and running, proceeding with master startup.
I20260812 21:06:18.289767 4215 sys_catalog.cc:455] T 00000000000000000000000000000000 P 95eb7458d397406299827f3ca14509ae [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "95eb7458d397406299827f3ca14509ae" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95eb7458d397406299827f3ca14509ae" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 39351 } } }
I20260812 21:06:18.290032 4215 sys_catalog.cc:458] T 00000000000000000000000000000000 P 95eb7458d397406299827f3ca14509ae [sys.catalog]: This master's current role is: LEADER
I20260812 21:06:18.289943 4217 sys_catalog.cc:455] T 00000000000000000000000000000000 P 95eb7458d397406299827f3ca14509ae [sys.catalog]: SysCatalogTable state changed. Reason: New leader 95eb7458d397406299827f3ca14509ae. Latest consensus state: current_term: 2 leader_uuid: "95eb7458d397406299827f3ca14509ae" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95eb7458d397406299827f3ca14509ae" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 39351 } } }
I20260812 21:06:18.290310 4217 sys_catalog.cc:458] T 00000000000000000000000000000000 P 95eb7458d397406299827f3ca14509ae [sys.catalog]: This master's current role is: LEADER
I20260812 21:06:18.290398 4231 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 21:06:18.291236 4231 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 21:06:18.291342 4231 catalog_manager.cc:1259] Loaded cluster ID: 0ae449a4aa65496abf11b8a33ff708f6
I20260812 21:06:18.291374 4231 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 21:06:18.291692 4231 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260812 21:06:18.294441 4231 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260812 21:06:18.296406 4231 catalog_manager.cc:1504] Loading token signing keys...
I20260812 21:06:18.296597 4231 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 95eb7458d397406299827f3ca14509ae: Loaded TSK: 0
I20260812 21:06:18.296792 4231 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 21:06:18.365137 4212 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:06:18.365345 4212 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:06:18.365367 4212 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:06:18.366591 4212 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:06:18.366637 4212 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:06:18.366664 4212 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.254.129
I20260812 21:06:18.368155 4212 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.254.148:41385
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.129.keytab
--principal=kudu/127.11.254.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.254.129:35389
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.254.129
--webserver_port=38073
--tserver_master_addrs=127.11.254.190:39351
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4212
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.254.129
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:06:18.368384 4212 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:06:18.368616 4212 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:06:18.373848 4262 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:06:18.373860 4260 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:06:18.373932 4265 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:06:18.374418 4212 server_base.cc:1034] running on GCE node
I20260812 21:06:18.374624 4212 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:06:18.374876 4212 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:06:18.376056 4212 hybrid_clock.cc:648] HybridClock initialized: now 1786568778376039 us; error 34 us; skew 500 ppm
Aug 12 21:06:18 dist-test-slave-vpvm krb5kdc[3199](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568778, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:06:18.378567 4212 init.cc:377] Logged in from keytab as kudu/127.11.254.129@KRBTEST.COM (short username kudu)
I20260812 21:06:18.379032 4212 webserver.cc:460] Webserver started at http://127.11.254.129:38073/ using document root <none> and password file <none>
I20260812 21:06:18.379278 4212 fs_manager.cc:359] Metadata directory not provided
I20260812 21:06:18.379336 4212 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:06:18.380748 4212 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 21:06:18.381415 4282 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:06:18.381608 4212 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 21:06:18.381719 4212 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/ts-0/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/ts-0/wal
uuid: "3ac4f761642144ab97f75d63be9843fd"
format_stamp: "Formatted at 2026-08-12 21:06:17 on dist-test-slave-vpvm"
server_key: "1f2072b5e42ba6bf4d89ffde48532cc7"
server_key_iv: "c1f4e2a6f8fbb9b8f6f50b7e63cda005"
server_key_version: "encryptionkey@0"
I20260812 21:06:18.382244 4212 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:06:18.393909 4212 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:06:18.396664 4212 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:06:18.396831 4212 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:06:18.397105 4212 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:06:18.397656 4290 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260812 21:06:18.399286 4212 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 21:06:18.399355 4212 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260812 21:06:18.399390 4212 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260812 21:06:18.400322 4212 ts_tablet_manager.cc:610] Registered 1 tablets
I20260812 21:06:18.400372 4212 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260812 21:06:18.400426 4290 tablet_bootstrap.cc:492] T 05c3b55e3da04169af9fd6b3a2ae4813 P 3ac4f761642144ab97f75d63be9843fd: Bootstrap starting.
I20260812 21:06:18.408062 4212 rpc_server.cc:307] RPC server started. Bound to: 127.11.254.129:35389
I20260812 21:06:18.408584 4212 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/ts-0/data/info.pb
I20260812 21:06:18.410200 4481 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.254.129:35389 every 8 connection(s)
I20260812 21:06:18.411510 4290 log.cc:824] T 05c3b55e3da04169af9fd6b3a2ae4813 P 3ac4f761642144ab97f75d63be9843fd: Log is configured to *not* fsync() on all Append() calls
I20260812 21:06:18.413784 4290 tablet_bootstrap.cc:492] T 05c3b55e3da04169af9fd6b3a2ae4813 P 3ac4f761642144ab97f75d63be9843fd: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 21:06:18.414134 4290 tablet_bootstrap.cc:492] T 05c3b55e3da04169af9fd6b3a2ae4813 P 3ac4f761642144ab97f75d63be9843fd: Bootstrap complete.
I20260812 21:06:18.415408 4290 ts_tablet_manager.cc:1397] T 05c3b55e3da04169af9fd6b3a2ae4813 P 3ac4f761642144ab97f75d63be9843fd: Time spent bootstrapping tablet: real 0.015s user 0.007s sys 0.007s
I20260812 21:06:18.416608 4290 raft_consensus.cc:348] T 05c3b55e3da04169af9fd6b3a2ae4813 P 3ac4f761642144ab97f75d63be9843fd [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fb1ffccc43e4fd9b9f078c167a38d55" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 46075 } } peers { permanent_uuid: "3ac4f761642144ab97f75d63be9843fd" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 35389 } } peers { permanent_uuid: "bbe9664da0ab4822b18a15afc541189b" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 33971 } }
I20260812 21:06:18.416935 4290 raft_consensus.cc:729] T 05c3b55e3da04169af9fd6b3a2ae4813 P 3ac4f761642144ab97f75d63be9843fd [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ac4f761642144ab97f75d63be9843fd, State: Initialized, Role: FOLLOWER
I20260812 21:06:18.417099 4290 consensus_queue.cc:260] T 05c3b55e3da04169af9fd6b3a2ae4813 P 3ac4f761642144ab97f75d63be9843fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fb1ffccc43e4fd9b9f078c167a38d55" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 46075 } } peers { permanent_uuid: "3ac4f761642144ab97f75d63be9843fd" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 35389 } } peers { permanent_uuid: "bbe9664da0ab4822b18a15afc541189b" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 33971 } }
I20260812 21:06:18.417359 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 4212
I20260812 21:06:18.417411 4290 ts_tablet_manager.cc:1428] T 05c3b55e3da04169af9fd6b3a2ae4813 P 3ac4f761642144ab97f75d63be9843fd: Time spent starting tablet: real 0.002s user 0.003s sys 0.001s
I20260812 21:06:18.417649 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.254.130:46075
--local_ip_for_outbound_sockets=127.11.254.130
--tserver_master_addrs=127.11.254.190:39351
--webserver_port=36717
--webserver_interface=127.11.254.130
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.130.keytab
--principal=kudu/127.11.254.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.254.148:41385
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 21:06:18 dist-test-slave-vpvm krb5kdc[3199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568778, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.129@KRBTEST.COM for kudu/127.11.254.190@KRBTEST.COM
I20260812 21:06:18.422345 4485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:18.409365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:39351 (local address 127.11.254.129:38113)
0812 21:06:18.410085 (+ 720us) negotiation.cc:107] Waiting for socket to connect
0812 21:06:18.410143 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0812 21:06:18.411021 (+ 878us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:06:18.411532 (+ 511us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:06:18.411537 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:06:18.411911 (+ 374us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:06:18.412133 (+ 222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:06:18.412140 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:06:18.413854 (+ 1714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:06:18.413855 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:06:18.415117 (+ 1262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:06:18.415121 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:06:18.415972 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:06:18.415974 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:06:18.416132 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:06:18.416778 (+ 646us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:06:18.416788 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:06:18.418352 (+ 1564us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:06:18.420463 (+ 2111us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:06:18.420471 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:06:18.420473 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:06:18.420786 (+ 313us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:06:18.421103 (+ 317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:06:18.421103 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:06:18.421104 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:06:18.421236 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:06:18.421674 (+ 438us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:06:18.421679 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:06:18.421838 (+ 159us) client_negotiation.cc:769] Sending connection context
0812 21:06:18.421933 (+ 95us) client_negotiation.cc:240] Negotiation successful
0812 21:06:18.422137 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":669,"thread_start_us":101,"threads_started":1}
I20260812 21:06:18.422664 4486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:18.409497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.129:38113 (local address 127.11.254.190:39351)
0812 21:06:18.409732 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0812 21:06:18.409742 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 21:06:18.410270 (+ 528us) server_negotiation.cc:407] Connection header received
0812 21:06:18.411269 (+ 999us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:06:18.411277 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:06:18.411335 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:06:18.411483 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:06:18.412270 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:06:18.413725 (+ 1455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:06:18.415217 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:06:18.415854 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:06:18.415971 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:06:18.418477 (+ 2506us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:06:18.418481 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:06:18.418482 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:06:18.418492 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:06:18.420364 (+ 1872us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:06:18.420891 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:06:18.420895 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:06:18.420895 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:06:18.421036 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:06:18.421306 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:06:18.421307 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:06:18.421307 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:06:18.421609 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:06:18.421688 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0812 21:06:18.422064 (+ 376us) server_negotiation.cc:299] Negotiation successful
0812 21:06:18.422245 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":179,"thread_start_us":88,"threads_started":1}
I20260812 21:06:18.423336 4483 heartbeater.cc:347] Connected to a master server at 127.11.254.190:39351
I20260812 21:06:18.423444 4483 heartbeater.cc:464] Registering TS with master...
I20260812 21:06:18.423661 4483 heartbeater.cc:511] Master 127.11.254.190:39351 requested a full tablet report, sending...
I20260812 21:06:18.424490 4155 ts_manager.cc:194] Registered new tserver with Master: 3ac4f761642144ab97f75d63be9843fd (127.11.254.129:35389)
I20260812 21:06:18.425675 4155 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.254.129@KRBTEST.COM'} at 127.11.254.129:38113
I20260812 21:06:18.426723 4483 heartbeater.cc:503] Master 127.11.254.190:39351 was elected leader, sending a full tablet report...
Aug 12 21:06:18 dist-test-slave-vpvm krb5kdc[3199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568778, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.190@KRBTEST.COM for kudu/127.11.254.129@KRBTEST.COM
I20260812 21:06:18.435865 4490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:18.425159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.129:35389 (local address 127.0.0.1:34894)
0812 21:06:18.425616 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0812 21:06:18.425660 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 21:06:18.425729 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:06:18.426354 (+ 625us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:06:18.426362 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:06:18.426719 (+ 357us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:06:18.426822 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:06:18.426972 (+ 150us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:06:18.428257 (+ 1285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:06:18.428259 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:06:18.429209 (+ 950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:06:18.429212 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:06:18.429872 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:06:18.429874 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:06:18.430002 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:06:18.430561 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:06:18.430567 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:06:18.432357 (+ 1790us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:06:18.434555 (+ 2198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:06:18.434563 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:06:18.434566 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:06:18.434832 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:06:18.435071 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:06:18.435072 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:06:18.435073 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:06:18.435164 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:06:18.435465 (+ 301us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:06:18.435472 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:06:18.435531 (+ 59us) client_negotiation.cc:769] Sending connection context
0812 21:06:18.435614 (+ 83us) client_negotiation.cc:240] Negotiation successful
0812 21:06:18.435751 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":422,"thread_start_us":98,"threads_started":1}
I20260812 21:06:18.435870 4492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:18.425430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34894 (local address 127.11.254.129:35389)
0812 21:06:18.425858 (+ 428us) server_negotiation.cc:206] Beginning negotiation
0812 21:06:18.425863 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 21:06:18.425872 (+ 9us) server_negotiation.cc:407] Connection header received
0812 21:06:18.426192 (+ 320us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:06:18.426195 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:06:18.426212 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:06:18.426282 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:06:18.427064 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:06:18.428175 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:06:18.429312 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:06:18.429786 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:06:18.429864 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:06:18.432459 (+ 2595us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:06:18.432470 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:06:18.432470 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:06:18.432479 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:06:18.434456 (+ 1977us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:06:18.434923 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:06:18.434927 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:06:18.434927 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:06:18.435002 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:06:18.435218 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:06:18.435218 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:06:18.435219 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:06:18.435409 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:06:18.435484 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 21:06:18.435657 (+ 173us) server_negotiation.cc:299] Negotiation successful
0812 21:06:18.435767 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":396,"thread_start_us":75,"threads_started":1}
I20260812 21:06:18.436653 4371 tablet_service.cc:1514] Processing DeleteTablet for tablet 05c3b55e3da04169af9fd6b3a2ae4813 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:06:17 UTC) from {username='kudu', principal='kudu/127.11.254.190@KRBTEST.COM'} at 127.0.0.1:34894
I20260812 21:06:18.437340 4493 tablet_replica.cc:330] stopping tablet replica
I20260812 21:06:18.437460 4493 raft_consensus.cc:2229] T 05c3b55e3da04169af9fd6b3a2ae4813 P 3ac4f761642144ab97f75d63be9843fd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 21:06:18.437531 4493 raft_consensus.cc:2258] T 05c3b55e3da04169af9fd6b3a2ae4813 P 3ac4f761642144ab97f75d63be9843fd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 21:06:18.437857 4493 ts_tablet_manager.cc:1905] T 05c3b55e3da04169af9fd6b3a2ae4813 P 3ac4f761642144ab97f75d63be9843fd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 21:06:18.440428 4493 ts_tablet_manager.cc:1918] T 05c3b55e3da04169af9fd6b3a2ae4813 P 3ac4f761642144ab97f75d63be9843fd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 21:06:18.440518 4493 log.cc:1196] T 05c3b55e3da04169af9fd6b3a2ae4813 P 3ac4f761642144ab97f75d63be9843fd: Deleting WAL directory at /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/ts-0/wal/wals/05c3b55e3da04169af9fd6b3a2ae4813
I20260812 21:06:18.440766 4493 ts_tablet_manager.cc:1939] T 05c3b55e3da04169af9fd6b3a2ae4813 P 3ac4f761642144ab97f75d63be9843fd: Deleting consensus metadata
I20260812 21:06:18.441362 4126 catalog_manager.cc:4841] TS 3ac4f761642144ab97f75d63be9843fd (127.11.254.129:35389): tablet 05c3b55e3da04169af9fd6b3a2ae4813 (table test-table [id=d8e06fb696f1440d83b7c25b311826d8]) successfully deleted
W20260812 21:06:18.506423 4489 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:06:18.506644 4489 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:06:18.506668 4489 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:06:18.507838 4489 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:06:18.507875 4489 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:06:18.507901 4489 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.254.130
I20260812 21:06:18.509398 4489 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.254.148:41385
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.130.keytab
--principal=kudu/127.11.254.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.254.130:46075
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.254.130
--webserver_port=36717
--tserver_master_addrs=127.11.254.190:39351
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4489
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.254.130
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:06:18.509608 4489 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:06:18.509804 4489 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:06:18.514808 4504 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:06:18.514814 4509 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:06:18.515062 4506 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:06:18.515053 4489 server_base.cc:1034] running on GCE node
I20260812 21:06:18.515303 4489 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:06:18.515528 4489 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:06:18.516666 4489 hybrid_clock.cc:648] HybridClock initialized: now 1786568778516648 us; error 32 us; skew 500 ppm
Aug 12 21:06:18 dist-test-slave-vpvm krb5kdc[3199](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568778, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:06:18.519181 4489 init.cc:377] Logged in from keytab as kudu/127.11.254.130@KRBTEST.COM (short username kudu)
I20260812 21:06:18.519613 4489 webserver.cc:460] Webserver started at http://127.11.254.130:36717/ using document root <none> and password file <none>
I20260812 21:06:18.519851 4489 fs_manager.cc:359] Metadata directory not provided
I20260812 21:06:18.519918 4489 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:06:18.521271 4489 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 21:06:18.521850 4524 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:06:18.522044 4489 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 21:06:18.522147 4489 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/ts-1/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/ts-1/wal
uuid: "5fb1ffccc43e4fd9b9f078c167a38d55"
format_stamp: "Formatted at 2026-08-12 21:06:17 on dist-test-slave-vpvm"
server_key: "2675fb8c3ba201c19c227f1b3b8df4f8"
server_key_iv: "e7f4afd28c563ca1cdba616336e9feef"
server_key_version: "encryptionkey@0"
I20260812 21:06:18.522491 4489 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:06:18.537865 4489 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:06:18.539757 4489 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:06:18.539901 4489 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:06:18.540124 4489 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:06:18.540426 4489 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:06:18.540460 4489 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:06:18.540491 4489 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:06:18.540515 4489 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:06:18.546226 4489 rpc_server.cc:307] RPC server started. Bound to: 127.11.254.130:46075
I20260812 21:06:18.546284 4693 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.254.130:46075 every 8 connection(s)
I20260812 21:06:18.546619 4489 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/ts-1/data/info.pb
Aug 12 21:06:18 dist-test-slave-vpvm krb5kdc[3199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568778, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.130@KRBTEST.COM for kudu/127.11.254.190@KRBTEST.COM
I20260812 21:06:18.554677 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 4489
I20260812 21:06:18.554905 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.254.131:33971
--local_ip_for_outbound_sockets=127.11.254.131
--tserver_master_addrs=127.11.254.190:39351
--webserver_port=40205
--webserver_interface=127.11.254.131
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.131.keytab
--principal=kudu/127.11.254.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.254.148:41385
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 21:06:18.555810 4486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:18.547302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.130:52211 (local address 127.11.254.190:39351)
0812 21:06:18.547374 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0812 21:06:18.547377 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 21:06:18.547580 (+ 203us) server_negotiation.cc:407] Connection header received
0812 21:06:18.548060 (+ 480us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:06:18.548062 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:06:18.548080 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:06:18.548134 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:06:18.549031 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:06:18.549555 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:06:18.550579 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:06:18.550791 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:06:18.550846 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:06:18.552864 (+ 2018us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:06:18.552866 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:06:18.552866 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:06:18.552873 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:06:18.554331 (+ 1458us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:06:18.554763 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:06:18.554765 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:06:18.554766 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:06:18.554832 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:06:18.555123 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:06:18.555125 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:06:18.555125 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:06:18.555265 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:06:18.555334 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0812 21:06:18.555629 (+ 295us) server_negotiation.cc:299] Negotiation successful
0812 21:06:18.555725 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260812 21:06:18.555902 4698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:18.547281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:39351 (local address 127.11.254.130:52211)
0812 21:06:18.547432 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0812 21:06:18.547469 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0812 21:06:18.547949 (+ 480us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:06:18.548190 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:06:18.548193 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:06:18.548748 (+ 555us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:06:18.548944 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:06:18.548949 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:06:18.549623 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:06:18.549624 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:06:18.550475 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:06:18.550478 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:06:18.550859 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:06:18.550860 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:06:18.550960 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:06:18.551376 (+ 416us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:06:18.551384 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:06:18.552782 (+ 1398us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:06:18.554424 (+ 1642us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:06:18.554432 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:06:18.554434 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:06:18.554673 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:06:18.554892 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:06:18.554893 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:06:18.554893 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:06:18.555019 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:06:18.555329 (+ 310us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:06:18.555334 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:06:18.555504 (+ 170us) client_negotiation.cc:769] Sending connection context
0812 21:06:18.555592 (+ 88us) client_negotiation.cc:240] Negotiation successful
0812 21:06:18.555724 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":85,"threads_started":1}
I20260812 21:06:18.556392 4694 heartbeater.cc:347] Connected to a master server at 127.11.254.190:39351
I20260812 21:06:18.556492 4694 heartbeater.cc:464] Registering TS with master...
I20260812 21:06:18.556707 4694 heartbeater.cc:511] Master 127.11.254.190:39351 requested a full tablet report, sending...
I20260812 21:06:18.557092 4155 ts_manager.cc:194] Registered new tserver with Master: 5fb1ffccc43e4fd9b9f078c167a38d55 (127.11.254.130:46075)
I20260812 21:06:18.557449 4155 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.254.130@KRBTEST.COM'} at 127.11.254.130:52211
W20260812 21:06:18.638222 4699 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:06:18.638427 4699 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:06:18.638461 4699 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:06:18.639637 4699 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:06:18.639678 4699 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:06:18.639704 4699 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.254.131
I20260812 21:06:18.641160 4699 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.254.148:41385
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.131.keytab
--principal=kudu/127.11.254.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.254.131:33971
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.254.131
--webserver_port=40205
--tserver_master_addrs=127.11.254.190:39351
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4699
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.254.131
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:06:18.641384 4699 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:06:18.641604 4699 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:06:18.646732 4706 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:06:18.646781 4708 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:06:18.646781 4705 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:06:18.646958 4699 server_base.cc:1034] running on GCE node
I20260812 21:06:18.647154 4699 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:06:18.647356 4699 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:06:18.648489 4699 hybrid_clock.cc:648] HybridClock initialized: now 1786568778648480 us; error 30 us; skew 500 ppm
Aug 12 21:06:18 dist-test-slave-vpvm krb5kdc[3199](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568778, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:06:18.651054 4699 init.cc:377] Logged in from keytab as kudu/127.11.254.131@KRBTEST.COM (short username kudu)
I20260812 21:06:18.651495 4699 webserver.cc:460] Webserver started at http://127.11.254.131:40205/ using document root <none> and password file <none>
I20260812 21:06:18.651695 4699 fs_manager.cc:359] Metadata directory not provided
I20260812 21:06:18.651753 4699 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:06:18.653061 4699 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 21:06:18.653694 4721 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:06:18.653870 4699 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 21:06:18.653998 4699 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/ts-2/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/ts-2/wal
uuid: "bbe9664da0ab4822b18a15afc541189b"
format_stamp: "Formatted at 2026-08-12 21:06:17 on dist-test-slave-vpvm"
server_key: "fef752db8fedde413b2ee07f590d945c"
server_key_iv: "aa8415e4b63fc4e866c2a11d014c8df3"
server_key_version: "encryptionkey@0"
I20260812 21:06:18.654366 4699 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:06:18.678403 4699 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:06:18.680474 4699 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:06:18.680599 4699 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:06:18.680836 4699 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:06:18.681110 4699 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:06:18.681143 4699 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:06:18.681164 4699 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:06:18.681174 4699 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:06:18.686931 4699 rpc_server.cc:307] RPC server started. Bound to: 127.11.254.131:33971
I20260812 21:06:18.686968 4904 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.254.131:33971 every 8 connection(s)
I20260812 21:06:18.687350 4699 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/ts-2/data/info.pb
I20260812 21:06:18.691587 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 4699
Aug 12 21:06:18 dist-test-slave-vpvm krb5kdc[3199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568778, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.131@KRBTEST.COM for kudu/127.11.254.190@KRBTEST.COM
I20260812 21:06:18.697397 4486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:18.688058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.131:58511 (local address 127.11.254.190:39351)
0812 21:06:18.688149 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0812 21:06:18.688151 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:06:18.688359 (+ 208us) server_negotiation.cc:407] Connection header received
0812 21:06:18.688886 (+ 527us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:06:18.688887 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:06:18.688898 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:06:18.688944 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:06:18.689918 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:06:18.690531 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:06:18.691561 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:06:18.691879 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:06:18.691942 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:06:18.694572 (+ 2630us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:06:18.694574 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:06:18.694574 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:06:18.694582 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:06:18.695922 (+ 1340us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:06:18.696324 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:06:18.696325 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:06:18.696326 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:06:18.696387 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:06:18.696652 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:06:18.696653 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:06:18.696653 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:06:18.696796 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:06:18.696849 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0812 21:06:18.697193 (+ 344us) server_negotiation.cc:299] Negotiation successful
0812 21:06:18.697289 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 21:06:18.697613 4910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:18.688048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:39351 (local address 127.11.254.131:58511)
0812 21:06:18.688243 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0812 21:06:18.688281 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 21:06:18.688803 (+ 522us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:06:18.689036 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:06:18.689041 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:06:18.689634 (+ 593us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:06:18.689837 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:06:18.689842 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:06:18.690618 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:06:18.690619 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:06:18.691459 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:06:18.691461 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:06:18.691998 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:06:18.691999 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:06:18.692195 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:06:18.692743 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:06:18.692755 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:06:18.694479 (+ 1724us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:06:18.695998 (+ 1519us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:06:18.696006 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:06:18.696010 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:06:18.696256 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:06:18.696434 (+ 178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:06:18.696435 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:06:18.696435 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:06:18.696587 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:06:18.696859 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:06:18.696866 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:06:18.697050 (+ 184us) client_negotiation.cc:769] Sending connection context
0812 21:06:18.697170 (+ 120us) client_negotiation.cc:240] Negotiation successful
0812 21:06:18.697361 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":86,"threads_started":1}
I20260812 21:06:18.698156 4906 heartbeater.cc:347] Connected to a master server at 127.11.254.190:39351
I20260812 21:06:18.698253 4906 heartbeater.cc:464] Registering TS with master...
I20260812 21:06:18.698446 4906 heartbeater.cc:511] Master 127.11.254.190:39351 requested a full tablet report, sending...
I20260812 21:06:18.698837 4155 ts_manager.cc:194] Registered new tserver with Master: bbe9664da0ab4822b18a15afc541189b (127.11.254.131:33971)
I20260812 21:06:18.698951 4911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:18.692074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41066 (local address 127.11.254.190:39351)
0812 21:06:18.692200 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0812 21:06:18.692202 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:06:18.692308 (+ 106us) server_negotiation.cc:407] Connection header received
0812 21:06:18.692382 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:06:18.692383 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:06:18.692397 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:06:18.692453 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:06:18.693265 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:06:18.693919 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:06:18.694803 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:06:18.695029 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:06:18.695098 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:06:18.696283 (+ 1185us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:06:18.696285 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:06:18.696285 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:06:18.696292 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:06:18.697734 (+ 1442us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:06:18.698189 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:06:18.698191 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:06:18.698191 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:06:18.698256 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:06:18.698433 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:06:18.698434 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:06:18.698434 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:06:18.698562 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:06:18.698745 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0812 21:06:18.698782 (+ 37us) server_negotiation.cc:299] Negotiation successful
0812 21:06:18.698861 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":68,"threads_started":1}
I20260812 21:06:18.699293 4155 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.254.131@KRBTEST.COM'} at 127.11.254.131:58511
I20260812 21:06:18.699626 12282 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 21:06:18.702394 4911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:18.700003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41078 (local address 127.11.254.190:39351)
0812 21:06:18.700038 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0812 21:06:18.700040 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:06:18.700161 (+ 121us) server_negotiation.cc:407] Connection header received
0812 21:06:18.700231 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:06:18.700231 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:06:18.700272 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:06:18.700330 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 21:06:18.700434 (+ 104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:06:18.700907 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:06:18.701633 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:06:18.701850 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:06:18.701904 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:06:18.702085 (+ 181us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 21:06:18.702188 (+ 103us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 21:06:18.702259 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0812 21:06:18.702295 (+ 36us) server_negotiation.cc:299] Negotiation successful
0812 21:06:18.702322 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260812 21:06:18.702750 4152 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:41078:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 21:06:18.704496 4152 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 21:06:18.707063 4371 tablet_service.cc:1467] Processing CreateTablet for tablet b4d484a48c5a434d9075392d3682b832 (DEFAULT_TABLE table=test-table [id=ffce520962cf450cb965f1850947d677]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:06:18.707283 4371 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b4d484a48c5a434d9075392d3682b832. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:06:18.708912 4290 tablet_bootstrap.cc:492] T b4d484a48c5a434d9075392d3682b832 P 3ac4f761642144ab97f75d63be9843fd: Bootstrap starting.
I20260812 21:06:18.709295 4290 tablet_bootstrap.cc:654] T b4d484a48c5a434d9075392d3682b832 P 3ac4f761642144ab97f75d63be9843fd: Neither blocks nor log segments found. Creating new log.
I20260812 21:06:18.710001 4290 tablet_bootstrap.cc:492] T b4d484a48c5a434d9075392d3682b832 P 3ac4f761642144ab97f75d63be9843fd: No bootstrap required, opened a new log
I20260812 21:06:18.710060 4290 ts_tablet_manager.cc:1397] T b4d484a48c5a434d9075392d3682b832 P 3ac4f761642144ab97f75d63be9843fd: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 21:06:18.710378 4490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:18.706478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.130:46075 (local address 127.0.0.1:56874)
0812 21:06:18.706537 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0812 21:06:18.706548 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 21:06:18.706589 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:06:18.707059 (+ 470us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:06:18.707060 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:06:18.707062 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:06:18.707115 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:06:18.707117 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:06:18.708254 (+ 1137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:06:18.708255 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:06:18.709160 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:06:18.709164 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:06:18.710123 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:06:18.710124 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:06:18.710200 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:06:18.710224 (+ 24us) client_negotiation.cc:769] Sending connection context
0812 21:06:18.710256 (+ 32us) client_negotiation.cc:240] Negotiation successful
0812 21:06:18.710304 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38,"mutex_wait_us":32}
I20260812 21:06:18.710394 4290 raft_consensus.cc:348] T b4d484a48c5a434d9075392d3682b832 P 3ac4f761642144ab97f75d63be9843fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fb1ffccc43e4fd9b9f078c167a38d55" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 46075 } } peers { permanent_uuid: "bbe9664da0ab4822b18a15afc541189b" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 33971 } } peers { permanent_uuid: "3ac4f761642144ab97f75d63be9843fd" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 35389 } }
I20260812 21:06:18.710471 4290 raft_consensus.cc:374] T b4d484a48c5a434d9075392d3682b832 P 3ac4f761642144ab97f75d63be9843fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:06:18.710528 4290 raft_consensus.cc:729] T b4d484a48c5a434d9075392d3682b832 P 3ac4f761642144ab97f75d63be9843fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ac4f761642144ab97f75d63be9843fd, State: Initialized, Role: FOLLOWER
I20260812 21:06:18.710582 4916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:18.706508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.131:33971 (local address 127.0.0.1:43520)
0812 21:06:18.706664 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0812 21:06:18.706674 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 21:06:18.706720 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:06:18.707090 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:06:18.707091 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:06:18.707094 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:06:18.707145 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:06:18.707146 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:06:18.708407 (+ 1261us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:06:18.708408 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:06:18.709461 (+ 1053us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:06:18.709464 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:06:18.710383 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:06:18.710384 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:06:18.710452 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:06:18.710456 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 21:06:18.710485 (+ 29us) client_negotiation.cc:240] Negotiation successful
0812 21:06:18.710520 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":96,"threads_started":1}
I20260812 21:06:18.710577 4290 consensus_queue.cc:260] T b4d484a48c5a434d9075392d3682b832 P 3ac4f761642144ab97f75d63be9843fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fb1ffccc43e4fd9b9f078c167a38d55" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 46075 } } peers { permanent_uuid: "bbe9664da0ab4822b18a15afc541189b" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 33971 } } peers { permanent_uuid: "3ac4f761642144ab97f75d63be9843fd" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 35389 } }
I20260812 21:06:18.710668 4290 ts_tablet_manager.cc:1428] T b4d484a48c5a434d9075392d3682b832 P 3ac4f761642144ab97f75d63be9843fd: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 21:06:18.710718 4917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:18.706688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56874 (local address 127.11.254.130:46075)
0812 21:06:18.706898 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0812 21:06:18.706903 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 21:06:18.706914 (+ 11us) server_negotiation.cc:407] Connection header received
0812 21:06:18.706950 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:06:18.706952 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:06:18.706965 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:06:18.707043 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:06:18.707177 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:06:18.708150 (+ 973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:06:18.709260 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:06:18.710024 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:06:18.710496 (+ 472us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:06:18.710570 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0812 21:06:18.710597 (+ 27us) server_negotiation.cc:299] Negotiation successful
0812 21:06:18.710642 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":128,"threads_started":1}
I20260812 21:06:18.710947 4919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:18.706735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43520 (local address 127.11.254.131:33971)
0812 21:06:18.706912 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0812 21:06:18.706917 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 21:06:18.706928 (+ 11us) server_negotiation.cc:407] Connection header received
0812 21:06:18.706966 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:06:18.706968 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:06:18.706983 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:06:18.707219 (+ 236us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:06:18.707286 (+ 67us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:06:18.708286 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:06:18.709561 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:06:18.710298 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:06:18.710737 (+ 439us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:06:18.710803 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0812 21:06:18.710828 (+ 25us) server_negotiation.cc:299] Negotiation successful
0812 21:06:18.710870 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":94,"threads_started":1}
I20260812 21:06:18.711608 4591 tablet_service.cc:1467] Processing CreateTablet for tablet b4d484a48c5a434d9075392d3682b832 (DEFAULT_TABLE table=test-table [id=ffce520962cf450cb965f1850947d677]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:06:18.711707 4794 tablet_service.cc:1467] Processing CreateTablet for tablet b4d484a48c5a434d9075392d3682b832 (DEFAULT_TABLE table=test-table [id=ffce520962cf450cb965f1850947d677]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:06:18.711928 4591 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b4d484a48c5a434d9075392d3682b832. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:06:18.712109 4794 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b4d484a48c5a434d9075392d3682b832. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:06:18.714916 4923 tablet_bootstrap.cc:492] T b4d484a48c5a434d9075392d3682b832 P bbe9664da0ab4822b18a15afc541189b: Bootstrap starting.
I20260812 21:06:18.715039 4921 tablet_bootstrap.cc:492] T b4d484a48c5a434d9075392d3682b832 P 5fb1ffccc43e4fd9b9f078c167a38d55: Bootstrap starting.
I20260812 21:06:18.715564 4923 tablet_bootstrap.cc:654] T b4d484a48c5a434d9075392d3682b832 P bbe9664da0ab4822b18a15afc541189b: Neither blocks nor log segments found. Creating new log.
I20260812 21:06:18.715634 4921 tablet_bootstrap.cc:654] T b4d484a48c5a434d9075392d3682b832 P 5fb1ffccc43e4fd9b9f078c167a38d55: Neither blocks nor log segments found. Creating new log.
I20260812 21:06:18.716030 4923 log.cc:824] T b4d484a48c5a434d9075392d3682b832 P bbe9664da0ab4822b18a15afc541189b: Log is configured to *not* fsync() on all Append() calls
I20260812 21:06:18.716135 4921 log.cc:824] T b4d484a48c5a434d9075392d3682b832 P 5fb1ffccc43e4fd9b9f078c167a38d55: Log is configured to *not* fsync() on all Append() calls
I20260812 21:06:18.716756 4923 tablet_bootstrap.cc:492] T b4d484a48c5a434d9075392d3682b832 P bbe9664da0ab4822b18a15afc541189b: No bootstrap required, opened a new log
I20260812 21:06:18.716826 4923 ts_tablet_manager.cc:1397] T b4d484a48c5a434d9075392d3682b832 P bbe9664da0ab4822b18a15afc541189b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 21:06:18.716864 4921 tablet_bootstrap.cc:492] T b4d484a48c5a434d9075392d3682b832 P 5fb1ffccc43e4fd9b9f078c167a38d55: No bootstrap required, opened a new log
I20260812 21:06:18.716933 4921 ts_tablet_manager.cc:1397] T b4d484a48c5a434d9075392d3682b832 P 5fb1ffccc43e4fd9b9f078c167a38d55: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 21:06:18.718359 4923 raft_consensus.cc:348] T b4d484a48c5a434d9075392d3682b832 P bbe9664da0ab4822b18a15afc541189b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fb1ffccc43e4fd9b9f078c167a38d55" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 46075 } } peers { permanent_uuid: "bbe9664da0ab4822b18a15afc541189b" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 33971 } } peers { permanent_uuid: "3ac4f761642144ab97f75d63be9843fd" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 35389 } }
I20260812 21:06:18.718536 4923 raft_consensus.cc:374] T b4d484a48c5a434d9075392d3682b832 P bbe9664da0ab4822b18a15afc541189b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:06:18.718444 4921 raft_consensus.cc:348] T b4d484a48c5a434d9075392d3682b832 P 5fb1ffccc43e4fd9b9f078c167a38d55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fb1ffccc43e4fd9b9f078c167a38d55" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 46075 } } peers { permanent_uuid: "bbe9664da0ab4822b18a15afc541189b" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 33971 } } peers { permanent_uuid: "3ac4f761642144ab97f75d63be9843fd" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 35389 } }
I20260812 21:06:18.718585 4923 raft_consensus.cc:729] T b4d484a48c5a434d9075392d3682b832 P bbe9664da0ab4822b18a15afc541189b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bbe9664da0ab4822b18a15afc541189b, State: Initialized, Role: FOLLOWER
I20260812 21:06:18.718601 4921 raft_consensus.cc:374] T b4d484a48c5a434d9075392d3682b832 P 5fb1ffccc43e4fd9b9f078c167a38d55 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:06:18.718632 4921 raft_consensus.cc:729] T b4d484a48c5a434d9075392d3682b832 P 5fb1ffccc43e4fd9b9f078c167a38d55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5fb1ffccc43e4fd9b9f078c167a38d55, State: Initialized, Role: FOLLOWER
I20260812 21:06:18.718705 4923 consensus_queue.cc:260] T b4d484a48c5a434d9075392d3682b832 P bbe9664da0ab4822b18a15afc541189b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fb1ffccc43e4fd9b9f078c167a38d55" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 46075 } } peers { permanent_uuid: "bbe9664da0ab4822b18a15afc541189b" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 33971 } } peers { permanent_uuid: "3ac4f761642144ab97f75d63be9843fd" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 35389 } }
I20260812 21:06:18.718743 4921 consensus_queue.cc:260] T b4d484a48c5a434d9075392d3682b832 P 5fb1ffccc43e4fd9b9f078c167a38d55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fb1ffccc43e4fd9b9f078c167a38d55" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 46075 } } peers { permanent_uuid: "bbe9664da0ab4822b18a15afc541189b" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 33971 } } peers { permanent_uuid: "3ac4f761642144ab97f75d63be9843fd" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 35389 } }
I20260812 21:06:18.719007 4921 ts_tablet_manager.cc:1428] T b4d484a48c5a434d9075392d3682b832 P 5fb1ffccc43e4fd9b9f078c167a38d55: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260812 21:06:18.719012 4923 ts_tablet_manager.cc:1428] T b4d484a48c5a434d9075392d3682b832 P bbe9664da0ab4822b18a15afc541189b: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260812 21:06:18.719043 4906 heartbeater.cc:503] Master 127.11.254.190:39351 was elected leader, sending a full tablet report...
I20260812 21:06:18.719297 4694 heartbeater.cc:503] Master 127.11.254.190:39351 was elected leader, sending a full tablet report...
I20260812 21:06:18.760317 4934 raft_consensus.cc:482] T b4d484a48c5a434d9075392d3682b832 P 5fb1ffccc43e4fd9b9f078c167a38d55 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 21:06:18.760450 4934 raft_consensus.cc:504] T b4d484a48c5a434d9075392d3682b832 P 5fb1ffccc43e4fd9b9f078c167a38d55 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fb1ffccc43e4fd9b9f078c167a38d55" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 46075 } } peers { permanent_uuid: "bbe9664da0ab4822b18a15afc541189b" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 33971 } } peers { permanent_uuid: "3ac4f761642144ab97f75d63be9843fd" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 35389 } }
I20260812 21:06:18.760849 4934 leader_election.cc:288] T b4d484a48c5a434d9075392d3682b832 P 5fb1ffccc43e4fd9b9f078c167a38d55 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bbe9664da0ab4822b18a15afc541189b (127.11.254.131:33971), 3ac4f761642144ab97f75d63be9843fd (127.11.254.129:35389)
I20260812 21:06:18.763690 4919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:18.761132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.130:46565 (local address 127.11.254.131:33971)
0812 21:06:18.761187 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0812 21:06:18.761189 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:06:18.761229 (+ 40us) server_negotiation.cc:407] Connection header received
0812 21:06:18.761258 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:06:18.761259 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:06:18.761269 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:06:18.761317 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:06:18.761477 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:06:18.762026 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:06:18.762956 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:06:18.763359 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:06:18.763422 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:06:18.763432 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0812 21:06:18.763577 (+ 145us) server_negotiation.cc:299] Negotiation successful
0812 21:06:18.763625 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260812 21:06:18.763689 4698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:18.761052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.131:33971 (local address 127.11.254.130:46565)
0812 21:06:18.761110 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0812 21:06:18.761123 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 21:06:18.761171 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:06:18.761336 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:06:18.761339 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:06:18.761341 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:06:18.761412 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:06:18.761414 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:06:18.762101 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:06:18.762102 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:06:18.762865 (+ 763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:06:18.762867 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:06:18.763438 (+ 571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:06:18.763440 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:06:18.763516 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:06:18.763519 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 21:06:18.763566 (+ 47us) client_negotiation.cc:240] Negotiation successful
0812 21:06:18.763610 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260812 21:06:18.764128 4492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:18.761283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.130:45697 (local address 127.11.254.129:35389)
0812 21:06:18.761330 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0812 21:06:18.761331 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 21:06:18.761449 (+ 118us) server_negotiation.cc:407] Connection header received
0812 21:06:18.761498 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:06:18.761500 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:06:18.761567 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:06:18.761622 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:06:18.761785 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:06:18.762311 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:06:18.763240 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:06:18.763667 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:06:18.763731 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:06:18.763817 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0812 21:06:18.764022 (+ 205us) server_negotiation.cc:299] Negotiation successful
0812 21:06:18.764068 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260812 21:06:18.764165 4936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:18.761200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.129:35389 (local address 127.11.254.130:45697)
0812 21:06:18.761383 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0812 21:06:18.761391 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0812 21:06:18.761440 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:06:18.761641 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:06:18.761642 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:06:18.761644 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:06:18.761711 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:06:18.761712 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:06:18.762390 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:06:18.762391 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:06:18.763151 (+ 760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:06:18.763154 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:06:18.763878 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:06:18.763880 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:06:18.763955 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:06:18.763960 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 21:06:18.764011 (+ 51us) client_negotiation.cc:240] Negotiation successful
0812 21:06:18.764063 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"mutex_wait_us":26,"thread_start_us":114,"threads_started":1}
I20260812 21:06:18.764165 4829 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4d484a48c5a434d9075392d3682b832" candidate_uuid: "5fb1ffccc43e4fd9b9f078c167a38d55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bbe9664da0ab4822b18a15afc541189b" is_pre_election: true
I20260812 21:06:18.764348 4829 raft_consensus.cc:2454] T b4d484a48c5a434d9075392d3682b832 P bbe9664da0ab4822b18a15afc541189b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5fb1ffccc43e4fd9b9f078c167a38d55 in term 0.
I20260812 21:06:18.764580 4525 leader_election.cc:302] T b4d484a48c5a434d9075392d3682b832 P 5fb1ffccc43e4fd9b9f078c167a38d55 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5fb1ffccc43e4fd9b9f078c167a38d55, bbe9664da0ab4822b18a15afc541189b; no voters:
I20260812 21:06:18.764701 4934 raft_consensus.cc:2781] T b4d484a48c5a434d9075392d3682b832 P 5fb1ffccc43e4fd9b9f078c167a38d55 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 21:06:18.764789 4934 raft_consensus.cc:482] T b4d484a48c5a434d9075392d3682b832 P 5fb1ffccc43e4fd9b9f078c167a38d55 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 21:06:18.764814 4934 raft_consensus.cc:3037] T b4d484a48c5a434d9075392d3682b832 P 5fb1ffccc43e4fd9b9f078c167a38d55 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:06:18.764889 4397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4d484a48c5a434d9075392d3682b832" candidate_uuid: "5fb1ffccc43e4fd9b9f078c167a38d55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ac4f761642144ab97f75d63be9843fd" is_pre_election: true
I20260812 21:06:18.765033 4397 raft_consensus.cc:2454] T b4d484a48c5a434d9075392d3682b832 P 3ac4f761642144ab97f75d63be9843fd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5fb1ffccc43e4fd9b9f078c167a38d55 in term 0.
I20260812 21:06:18.765672 4934 raft_consensus.cc:504] T b4d484a48c5a434d9075392d3682b832 P 5fb1ffccc43e4fd9b9f078c167a38d55 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fb1ffccc43e4fd9b9f078c167a38d55" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 46075 } } peers { permanent_uuid: "bbe9664da0ab4822b18a15afc541189b" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 33971 } } peers { permanent_uuid: "3ac4f761642144ab97f75d63be9843fd" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 35389 } }
I20260812 21:06:18.765798 4934 leader_election.cc:288] T b4d484a48c5a434d9075392d3682b832 P 5fb1ffccc43e4fd9b9f078c167a38d55 [CANDIDATE]: Term 1 election: Requested vote from peers bbe9664da0ab4822b18a15afc541189b (127.11.254.131:33971), 3ac4f761642144ab97f75d63be9843fd (127.11.254.129:35389)
I20260812 21:06:18.766084 4829 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4d484a48c5a434d9075392d3682b832" candidate_uuid: "5fb1ffccc43e4fd9b9f078c167a38d55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bbe9664da0ab4822b18a15afc541189b"
I20260812 21:06:18.766090 4397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4d484a48c5a434d9075392d3682b832" candidate_uuid: "5fb1ffccc43e4fd9b9f078c167a38d55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ac4f761642144ab97f75d63be9843fd"
I20260812 21:06:18.766168 4829 raft_consensus.cc:3037] T b4d484a48c5a434d9075392d3682b832 P bbe9664da0ab4822b18a15afc541189b [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:06:18.766170 4397 raft_consensus.cc:3037] T b4d484a48c5a434d9075392d3682b832 P 3ac4f761642144ab97f75d63be9843fd [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:06:18.766860 4397 raft_consensus.cc:2454] T b4d484a48c5a434d9075392d3682b832 P 3ac4f761642144ab97f75d63be9843fd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5fb1ffccc43e4fd9b9f078c167a38d55 in term 1.
I20260812 21:06:18.766968 4829 raft_consensus.cc:2454] T b4d484a48c5a434d9075392d3682b832 P bbe9664da0ab4822b18a15afc541189b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5fb1ffccc43e4fd9b9f078c167a38d55 in term 1.
I20260812 21:06:18.767007 4525 leader_election.cc:302] T b4d484a48c5a434d9075392d3682b832 P 5fb1ffccc43e4fd9b9f078c167a38d55 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ac4f761642144ab97f75d63be9843fd, 5fb1ffccc43e4fd9b9f078c167a38d55; no voters:
I20260812 21:06:18.767128 4934 raft_consensus.cc:2781] T b4d484a48c5a434d9075392d3682b832 P 5fb1ffccc43e4fd9b9f078c167a38d55 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:06:18.767300 4934 raft_consensus.cc:686] T b4d484a48c5a434d9075392d3682b832 P 5fb1ffccc43e4fd9b9f078c167a38d55 [term 1 LEADER]: Becoming Leader. State: Replica: 5fb1ffccc43e4fd9b9f078c167a38d55, State: Running, Role: LEADER
I20260812 21:06:18.767395 4934 consensus_queue.cc:237] T b4d484a48c5a434d9075392d3682b832 P 5fb1ffccc43e4fd9b9f078c167a38d55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fb1ffccc43e4fd9b9f078c167a38d55" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 46075 } } peers { permanent_uuid: "bbe9664da0ab4822b18a15afc541189b" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 33971 } } peers { permanent_uuid: "3ac4f761642144ab97f75d63be9843fd" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 35389 } }
I20260812 21:06:18.768298 4152 catalog_manager.cc:5495] T b4d484a48c5a434d9075392d3682b832 P 5fb1ffccc43e4fd9b9f078c167a38d55 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5fb1ffccc43e4fd9b9f078c167a38d55 (127.11.254.130). New cstate: current_term: 1 leader_uuid: "5fb1ffccc43e4fd9b9f078c167a38d55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fb1ffccc43e4fd9b9f078c167a38d55" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 46075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bbe9664da0ab4822b18a15afc541189b" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 33971 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ac4f761642144ab97f75d63be9843fd" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 35389 } health_report { overall_health: UNKNOWN } } }
I20260812 21:06:18.784067 4917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:18.781669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56890 (local address 127.11.254.130:46075)
0812 21:06:18.781724 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0812 21:06:18.781726 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:06:18.781737 (+ 11us) server_negotiation.cc:407] Connection header received
0812 21:06:18.781847 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:06:18.781848 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:06:18.781867 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:06:18.781925 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 21:06:18.782072 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:06:18.782564 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:06:18.783314 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:06:18.783505 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:06:18.783566 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:06:18.783717 (+ 151us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 21:06:18.783825 (+ 108us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 21:06:18.783899 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0812 21:06:18.783919 (+ 20us) server_negotiation.cc:299] Negotiation successful
0812 21:06:18.783942 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260812 21:06:18.785990 4829 raft_consensus.cc:1261] T b4d484a48c5a434d9075392d3682b832 P bbe9664da0ab4822b18a15afc541189b [term 1 FOLLOWER]: Refusing update from remote peer 5fb1ffccc43e4fd9b9f078c167a38d55: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 21:06:18.785987 4397 raft_consensus.cc:1261] T b4d484a48c5a434d9075392d3682b832 P 3ac4f761642144ab97f75d63be9843fd [term 1 FOLLOWER]: Refusing update from remote peer 5fb1ffccc43e4fd9b9f078c167a38d55: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 21:06:18.786332 4934 consensus_queue.cc:1035] T b4d484a48c5a434d9075392d3682b832 P 5fb1ffccc43e4fd9b9f078c167a38d55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bbe9664da0ab4822b18a15afc541189b" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 33971 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 21:06:18.786461 4939 consensus_queue.cc:1035] T b4d484a48c5a434d9075392d3682b832 P 5fb1ffccc43e4fd9b9f078c167a38d55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ac4f761642144ab97f75d63be9843fd" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 35389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 21:06:18.789167 4945 mvcc.cc:205] Tried to move back new op lower bound from 7317785717904498688 to 7317785717831987200. Current Snapshot: MvccSnapshot[applied={T|T < 7317785717904498688}]
I20260812 21:06:18.789475 4946 mvcc.cc:205] Tried to move back new op lower bound from 7317785717904498688 to 7317785717831987200. Current Snapshot: MvccSnapshot[applied={T|T < 7317785717904498688}]
I20260812 21:06:18.791095 4155 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:41078:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 21:06:18.791177 4155 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:41078:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 21:06:18.791891 4155 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 95eb7458d397406299827f3ca14509ae: Sending DeleteTablet for 3 replicas of tablet b4d484a48c5a434d9075392d3682b832
I20260812 21:06:18.792193 4794 tablet_service.cc:1514] Processing DeleteTablet for tablet b4d484a48c5a434d9075392d3682b832 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:06:18 UTC) from {username='kudu', principal='kudu/127.11.254.190@KRBTEST.COM'} at 127.0.0.1:43520
I20260812 21:06:18.792272 4371 tablet_service.cc:1514] Processing DeleteTablet for tablet b4d484a48c5a434d9075392d3682b832 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:06:18 UTC) from {username='kudu', principal='kudu/127.11.254.190@KRBTEST.COM'} at 127.0.0.1:34894
I20260812 21:06:18.792285 4591 tablet_service.cc:1514] Processing DeleteTablet for tablet b4d484a48c5a434d9075392d3682b832 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:06:18 UTC) from {username='kudu', principal='kudu/127.11.254.190@KRBTEST.COM'} at 127.0.0.1:56874
I20260812 21:06:18.792372 4493 tablet_replica.cc:330] T b4d484a48c5a434d9075392d3682b832 P 3ac4f761642144ab97f75d63be9843fd: stopping tablet replica
I20260812 21:06:18.792429 4493 raft_consensus.cc:2229] T b4d484a48c5a434d9075392d3682b832 P 3ac4f761642144ab97f75d63be9843fd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 21:06:18.792477 4493 raft_consensus.cc:2258] T b4d484a48c5a434d9075392d3682b832 P 3ac4f761642144ab97f75d63be9843fd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 21:06:18.792490 4958 tablet_replica.cc:330] T b4d484a48c5a434d9075392d3682b832 P 5fb1ffccc43e4fd9b9f078c167a38d55: stopping tablet replica
I20260812 21:06:18.792578 4958 raft_consensus.cc:2229] T b4d484a48c5a434d9075392d3682b832 P 5fb1ffccc43e4fd9b9f078c167a38d55 [term 1 LEADER]: Raft consensus shutting down.
I20260812 21:06:18.792608 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 4212
I20260812 21:06:18.792680 4958 raft_consensus.cc:2258] T b4d484a48c5a434d9075392d3682b832 P 5fb1ffccc43e4fd9b9f078c167a38d55 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 21:06:18.801882 4958 ts_tablet_manager.cc:1905] T b4d484a48c5a434d9075392d3682b832 P 5fb1ffccc43e4fd9b9f078c167a38d55: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 21:06:18.803732 4958 ts_tablet_manager.cc:1918] T b4d484a48c5a434d9075392d3682b832 P 5fb1ffccc43e4fd9b9f078c167a38d55: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 21:06:18.803799 4958 log.cc:1196] T b4d484a48c5a434d9075392d3682b832 P 5fb1ffccc43e4fd9b9f078c167a38d55: Deleting WAL directory at /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/ts-1/wal/wals/b4d484a48c5a434d9075392d3682b832
I20260812 21:06:18.804031 4958 ts_tablet_manager.cc:1939] T b4d484a48c5a434d9075392d3682b832 P 5fb1ffccc43e4fd9b9f078c167a38d55: Deleting consensus metadata
I20260812 21:06:18.804468 4129 catalog_manager.cc:4841] TS 5fb1ffccc43e4fd9b9f078c167a38d55 (127.11.254.130:46075): tablet b4d484a48c5a434d9075392d3682b832 (table test-table [id=ffce520962cf450cb965f1850947d677]) successfully deleted
I20260812 21:06:18.809262 4956 tablet_replica.cc:330] T b4d484a48c5a434d9075392d3682b832 P bbe9664da0ab4822b18a15afc541189b: stopping tablet replica
I20260812 21:06:18.809553 4956 raft_consensus.cc:2229] T b4d484a48c5a434d9075392d3682b832 P bbe9664da0ab4822b18a15afc541189b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 21:06:18.809713 4956 raft_consensus.cc:2258] T b4d484a48c5a434d9075392d3682b832 P bbe9664da0ab4822b18a15afc541189b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 21:06:18.811923 4956 ts_tablet_manager.cc:1905] T b4d484a48c5a434d9075392d3682b832 P bbe9664da0ab4822b18a15afc541189b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 21:06:18.813690 4956 ts_tablet_manager.cc:1918] T b4d484a48c5a434d9075392d3682b832 P bbe9664da0ab4822b18a15afc541189b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 21:06:18.813782 4956 log.cc:1196] T b4d484a48c5a434d9075392d3682b832 P bbe9664da0ab4822b18a15afc541189b: Deleting WAL directory at /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/ts-2/wal/wals/b4d484a48c5a434d9075392d3682b832
I20260812 21:06:18.814116 4956 ts_tablet_manager.cc:1939] T b4d484a48c5a434d9075392d3682b832 P bbe9664da0ab4822b18a15afc541189b: Deleting consensus metadata
I20260812 21:06:18.814687 4126 catalog_manager.cc:4841] TS bbe9664da0ab4822b18a15afc541189b (127.11.254.131:33971): tablet b4d484a48c5a434d9075392d3682b832 (table test-table [id=ffce520962cf450cb965f1850947d677]) successfully deleted
W20260812 21:06:18.816495 4126 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.254.129:35389 (error 108)
I20260812 21:06:18.816694 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 4489
I20260812 21:06:18.852980 4916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:18.852773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.129:35389 (local address 127.0.0.1:50768)
0812 21:06:18.852850 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0812 21:06:18.852928 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.129:35389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260812 21:06:18.853832 4126 catalog_manager.cc:4584] TS 3ac4f761642144ab97f75d63be9843fd (127.11.254.129:35389): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b4d484a48c5a434d9075392d3682b832: Network error: Client connection negotiation failed: client connection to 127.11.254.129:35389: connect: Connection refused (error 111)
I20260812 21:06:18.857903 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 4699
I20260812 21:06:18.877674 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 4099
I20260812 21:06:18.890969 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.254.190:39351
--webserver_interface=127.11.254.190
--webserver_port=41491
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.190.keytab
--principal=kudu/127.11.254.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.254.148:41385
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.254.190:39351
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:06:18.973485 4963 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:06:18.973707 4963 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:06:18.973735 4963 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:06:18.974924 4963 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 21:06:18.974970 4963 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:06:18.974989 4963 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:06:18.975005 4963 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 21:06:18.975020 4963 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 21:06:18.976949 4963 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.254.148:41385
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.254.190:39351
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.190.keytab
--principal=kudu/127.11.254.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.254.190:39351
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.254.190
--webserver_port=41491
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4963
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:06:18.977240 4963 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:06:18.977506 4963 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:06:18.982344 4975 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:06:18.982344 4977 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:06:18.982393 4970 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:06:18.982672 4963 server_base.cc:1034] running on GCE node
I20260812 21:06:18.982905 4963 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:06:18.983215 4963 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:06:18.984370 4963 hybrid_clock.cc:648] HybridClock initialized: now 1786568778984339 us; error 52 us; skew 500 ppm
Aug 12 21:06:18 dist-test-slave-vpvm krb5kdc[3199](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568778, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:06:18.987005 4963 init.cc:377] Logged in from keytab as kudu/127.11.254.190@KRBTEST.COM (short username kudu)
I20260812 21:06:18.987460 4963 webserver.cc:460] Webserver started at http://127.11.254.190:41491/ using document root <none> and password file <none>
I20260812 21:06:18.987663 4963 fs_manager.cc:359] Metadata directory not provided
I20260812 21:06:18.987704 4963 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:06:18.989107 4963 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 21:06:18.989880 4989 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:06:18.990087 4963 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 21:06:18.990180 4963 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/master-0/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/master-0/wal
uuid: "95eb7458d397406299827f3ca14509ae"
format_stamp: "Formatted at 2026-08-12 21:06:17 on dist-test-slave-vpvm"
server_key: "cf81d91cdcb203811592b175a7bc3fc7"
server_key_iv: "a1bef8cd145b42ee2459799f78eca1da"
server_key_version: "encryptionkey@0"
I20260812 21:06:18.990608 4963 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:06:18.998484 4963 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:06:19.000458 4963 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:06:19.000572 4963 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:06:19.004730 4963 rpc_server.cc:307] RPC server started. Bound to: 127.11.254.190:39351
I20260812 21:06:19.004765 5077 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.254.190:39351 every 8 connection(s)
I20260812 21:06:19.005133 4963 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
I20260812 21:06:19.006354 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 4963
I20260812 21:06:19.006659 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.254.129:35389
--local_ip_for_outbound_sockets=127.11.254.129
--tserver_master_addrs=127.11.254.190:39351
--webserver_port=38073
--webserver_interface=127.11.254.129
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.129.keytab
--principal=kudu/127.11.254.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.254.148:41385
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 21:06:19.006909 5082 sys_catalog.cc:263] Verifying existing consensus state
I20260812 21:06:19.007596 5082 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 95eb7458d397406299827f3ca14509ae: Bootstrap starting.
I20260812 21:06:19.018795 5082 log.cc:824] T 00000000000000000000000000000000 P 95eb7458d397406299827f3ca14509ae: Log is configured to *not* fsync() on all Append() calls
I20260812 21:06:19.021919 5082 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 95eb7458d397406299827f3ca14509ae: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 21:06:19.022341 5082 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 95eb7458d397406299827f3ca14509ae: Bootstrap complete.
I20260812 21:06:19.025717 5082 raft_consensus.cc:348] T 00000000000000000000000000000000 P 95eb7458d397406299827f3ca14509ae [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95eb7458d397406299827f3ca14509ae" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 39351 } }
I20260812 21:06:19.026162 5082 raft_consensus.cc:729] T 00000000000000000000000000000000 P 95eb7458d397406299827f3ca14509ae [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95eb7458d397406299827f3ca14509ae, State: Initialized, Role: FOLLOWER
I20260812 21:06:19.026351 5082 consensus_queue.cc:260] T 00000000000000000000000000000000 P 95eb7458d397406299827f3ca14509ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95eb7458d397406299827f3ca14509ae" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 39351 } }
I20260812 21:06:19.026424 5082 raft_consensus.cc:388] T 00000000000000000000000000000000 P 95eb7458d397406299827f3ca14509ae [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 21:06:19.026469 5082 raft_consensus.cc:482] T 00000000000000000000000000000000 P 95eb7458d397406299827f3ca14509ae [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 21:06:19.026517 5082 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 95eb7458d397406299827f3ca14509ae [term 2 FOLLOWER]: Advancing to term 3
I20260812 21:06:19.027673 5082 raft_consensus.cc:504] T 00000000000000000000000000000000 P 95eb7458d397406299827f3ca14509ae [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95eb7458d397406299827f3ca14509ae" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 39351 } }
I20260812 21:06:19.027767 5082 leader_election.cc:302] T 00000000000000000000000000000000 P 95eb7458d397406299827f3ca14509ae [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 95eb7458d397406299827f3ca14509ae; no voters:
I20260812 21:06:19.027951 5082 leader_election.cc:288] T 00000000000000000000000000000000 P 95eb7458d397406299827f3ca14509ae [CANDIDATE]: Term 3 election: Requested vote from peers
I20260812 21:06:19.028016 5090 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 95eb7458d397406299827f3ca14509ae [term 3 FOLLOWER]: Leader election won for term 3
I20260812 21:06:19.028141 5090 raft_consensus.cc:686] T 00000000000000000000000000000000 P 95eb7458d397406299827f3ca14509ae [term 3 LEADER]: Becoming Leader. State: Replica: 95eb7458d397406299827f3ca14509ae, State: Running, Role: LEADER
I20260812 21:06:19.028236 5082 sys_catalog.cc:564] T 00000000000000000000000000000000 P 95eb7458d397406299827f3ca14509ae [sys.catalog]: configured and running, proceeding with master startup.
I20260812 21:06:19.028229 5090 consensus_queue.cc:237] T 00000000000000000000000000000000 P 95eb7458d397406299827f3ca14509ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95eb7458d397406299827f3ca14509ae" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 39351 } }
I20260812 21:06:19.028558 5093 sys_catalog.cc:455] T 00000000000000000000000000000000 P 95eb7458d397406299827f3ca14509ae [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "95eb7458d397406299827f3ca14509ae" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95eb7458d397406299827f3ca14509ae" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 39351 } } }
I20260812 21:06:19.028635 5093 sys_catalog.cc:458] T 00000000000000000000000000000000 P 95eb7458d397406299827f3ca14509ae [sys.catalog]: This master's current role is: LEADER
I20260812 21:06:19.028947 5094 sys_catalog.cc:455] T 00000000000000000000000000000000 P 95eb7458d397406299827f3ca14509ae [sys.catalog]: SysCatalogTable state changed. Reason: New leader 95eb7458d397406299827f3ca14509ae. Latest consensus state: current_term: 3 leader_uuid: "95eb7458d397406299827f3ca14509ae" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95eb7458d397406299827f3ca14509ae" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 39351 } } }
I20260812 21:06:19.029016 5094 sys_catalog.cc:458] T 00000000000000000000000000000000 P 95eb7458d397406299827f3ca14509ae [sys.catalog]: This master's current role is: LEADER
I20260812 21:06:19.029150 5098 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 21:06:19.030086 5098 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 21:06:19.030220 5098 catalog_manager.cc:1259] Loaded cluster ID: 0ae449a4aa65496abf11b8a33ff708f6
I20260812 21:06:19.030292 5098 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260812 21:06:19.030714 5098 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fb4f0bda9ad google::LogMessage::Fail() at ??:0
@ 0x7fb4f0bdd88a google::LogMessage::SendToLog() at ??:0
@ 0x7fb4f0bda4de google::LogMessage::Flush() at ??:0
@ 0x7fb4f0bdba19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fb4f8afb0f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7fb4f8b048b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fb4f2aa80d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fb4f2a9d954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7fb4f39016db start_thread at ??:0
@ 0x7fb4ef6d861f clone at ??:0
W20260812 21:06:19.106267 5084 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:06:19.106477 5084 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:06:19.106511 5084 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:06:19.107676 5084 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:06:19.107717 5084 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:06:19.107751 5084 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.254.129
I20260812 21:06:19.109210 5084 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.254.148:41385
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.129.keytab
--principal=kudu/127.11.254.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.254.129:35389
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.254.129
--webserver_port=38073
--tserver_master_addrs=127.11.254.190:39351
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5084
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.254.129
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:06:19.109450 5084 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:06:19.109661 5084 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:06:19.114698 5145 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:06:19.114795 5142 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:06:19.114711 5139 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:06:19.114924 5084 server_base.cc:1034] running on GCE node
I20260812 21:06:19.115173 5084 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:06:19.115401 5084 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:06:19.116557 5084 hybrid_clock.cc:648] HybridClock initialized: now 1786568779116535 us; error 58 us; skew 500 ppm
Aug 12 21:06:19 dist-test-slave-vpvm krb5kdc[3199](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568779, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:06:19.119095 5084 init.cc:377] Logged in from keytab as kudu/127.11.254.129@KRBTEST.COM (short username kudu)
I20260812 21:06:19.119530 5084 webserver.cc:460] Webserver started at http://127.11.254.129:38073/ using document root <none> and password file <none>
I20260812 21:06:19.119758 5084 fs_manager.cc:359] Metadata directory not provided
I20260812 21:06:19.119808 5084 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:06:19.121201 5084 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 21:06:19.121980 5153 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:06:19.122154 5084 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 21:06:19.122233 5084 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/ts-0/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/ts-0/wal
uuid: "3ac4f761642144ab97f75d63be9843fd"
format_stamp: "Formatted at 2026-08-12 21:06:17 on dist-test-slave-vpvm"
server_key: "1f2072b5e42ba6bf4d89ffde48532cc7"
server_key_iv: "c1f4e2a6f8fbb9b8f6f50b7e63cda005"
server_key_version: "encryptionkey@0"
I20260812 21:06:19.122623 5084 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:06:19.142589 5084 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:06:19.144932 5084 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:06:19.145083 5084 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:06:19.145320 5084 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:06:19.145776 5165 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260812 21:06:19.147902 5084 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 21:06:19.147961 5084 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260812 21:06:19.147998 5084 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260812 21:06:19.148707 5084 ts_tablet_manager.cc:610] Registered 1 tablets
I20260812 21:06:19.148742 5084 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260812 21:06:19.148841 5165 tablet_bootstrap.cc:492] T b4d484a48c5a434d9075392d3682b832 P 3ac4f761642144ab97f75d63be9843fd: Bootstrap starting.
I20260812 21:06:19.155632 5084 rpc_server.cc:307] RPC server started. Bound to: 127.11.254.129:35389
I20260812 21:06:19.156031 5084 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/ts-0/data/info.pb
I20260812 21:06:19.156605 5331 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.254.129:35389 every 8 connection(s)
I20260812 21:06:19.160823 5165 log.cc:824] T b4d484a48c5a434d9075392d3682b832 P 3ac4f761642144ab97f75d63be9843fd: Log is configured to *not* fsync() on all Append() calls
I20260812 21:06:19.163414 5165 tablet_bootstrap.cc:492] T b4d484a48c5a434d9075392d3682b832 P 3ac4f761642144ab97f75d63be9843fd: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 21:06:19.163583 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 5084
I20260812 21:06:19.163743 5165 tablet_bootstrap.cc:492] T b4d484a48c5a434d9075392d3682b832 P 3ac4f761642144ab97f75d63be9843fd: Bootstrap complete.
I20260812 21:06:19.163892 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.254.130:46075
--local_ip_for_outbound_sockets=127.11.254.130
--tserver_master_addrs=127.11.254.190:39351
--webserver_port=36717
--webserver_interface=127.11.254.130
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.130.keytab
--principal=kudu/127.11.254.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.254.148:41385
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 21:06:19.164698 5165 ts_tablet_manager.cc:1397] T b4d484a48c5a434d9075392d3682b832 P 3ac4f761642144ab97f75d63be9843fd: Time spent bootstrapping tablet: real 0.016s user 0.005s sys 0.008s
I20260812 21:06:19.165967 5165 raft_consensus.cc:348] T b4d484a48c5a434d9075392d3682b832 P 3ac4f761642144ab97f75d63be9843fd [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fb1ffccc43e4fd9b9f078c167a38d55" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 46075 } } peers { permanent_uuid: "bbe9664da0ab4822b18a15afc541189b" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 33971 } } peers { permanent_uuid: "3ac4f761642144ab97f75d63be9843fd" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 35389 } }
I20260812 21:06:19.166316 5165 raft_consensus.cc:729] T b4d484a48c5a434d9075392d3682b832 P 3ac4f761642144ab97f75d63be9843fd [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ac4f761642144ab97f75d63be9843fd, State: Initialized, Role: FOLLOWER
I20260812 21:06:19.166481 5165 consensus_queue.cc:260] T b4d484a48c5a434d9075392d3682b832 P 3ac4f761642144ab97f75d63be9843fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fb1ffccc43e4fd9b9f078c167a38d55" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 46075 } } peers { permanent_uuid: "bbe9664da0ab4822b18a15afc541189b" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 33971 } } peers { permanent_uuid: "3ac4f761642144ab97f75d63be9843fd" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 35389 } }
I20260812 21:06:19.166811 5165 ts_tablet_manager.cc:1428] T b4d484a48c5a434d9075392d3682b832 P 3ac4f761642144ab97f75d63be9843fd: Time spent starting tablet: real 0.002s user 0.003s sys 0.002s
W20260812 21:06:19.227125 5337 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:06:19.156796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:39351 (local address 127.11.254.129:49817)
0812 21:06:19.157033 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0812 21:06:19.157081 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 21:06:19.157559 (+ 478us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:06:19.226939 (+ 69380us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.190:39351: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":104,"threads_started":1}
W20260812 21:06:19.227351 5332 heartbeater.cc:644] Failed to heartbeat to 127.11.254.190:39351 (0 consecutive failures): Network error: Failed to ping master at 127.11.254.190:39351: Client connection negotiation failed: client connection to 127.11.254.190:39351: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260812 21:06:19.227712 5337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:19.227590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:39351 (local address 127.11.254.129:53209)
0812 21:06:19.227644 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0812 21:06:19.227686 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.190:39351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260812 21:06:19.228075 5337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:19.227982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:39351 (local address 127.11.254.129:58769)
0812 21:06:19.228032 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0812 21:06:19.228055 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.190:39351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260812 21:06:19.228219 5332 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 21:06:19.255136 5344 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:06:19.255458 5344 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:06:19.255498 5344 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:06:19.257652 5344 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:06:19.257738 5344 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:06:19.257795 5344 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.254.130
I20260812 21:06:19.260038 5344 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.254.148:41385
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.130.keytab
--principal=kudu/127.11.254.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.254.130:46075
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.254.130
--webserver_port=36717
--tserver_master_addrs=127.11.254.190:39351
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5344
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.254.130
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:06:19.260360 5344 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:06:19.260656 5344 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:06:19.267758 5354 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:06:19.267958 5353 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:06:19.268159 5357 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:06:19.268121 5344 server_base.cc:1034] running on GCE node
I20260812 21:06:19.268517 5344 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:06:19.268798 5344 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:06:19.270005 5344 hybrid_clock.cc:648] HybridClock initialized: now 1786568779269981 us; error 38 us; skew 500 ppm
Aug 12 21:06:19 dist-test-slave-vpvm krb5kdc[3199](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568779, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:06:19.273176 5344 init.cc:377] Logged in from keytab as kudu/127.11.254.130@KRBTEST.COM (short username kudu)
I20260812 21:06:19.273710 5344 webserver.cc:460] Webserver started at http://127.11.254.130:36717/ using document root <none> and password file <none>
I20260812 21:06:19.273941 5344 fs_manager.cc:359] Metadata directory not provided
I20260812 21:06:19.274045 5344 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:06:19.275511 5344 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 21:06:19.281224 5368 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:06:19.281416 5344 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.000s sys 0.004s
I20260812 21:06:19.281524 5344 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/ts-1/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/ts-1/wal
uuid: "5fb1ffccc43e4fd9b9f078c167a38d55"
format_stamp: "Formatted at 2026-08-12 21:06:17 on dist-test-slave-vpvm"
server_key: "2675fb8c3ba201c19c227f1b3b8df4f8"
server_key_iv: "e7f4afd28c563ca1cdba616336e9feef"
server_key_version: "encryptionkey@0"
I20260812 21:06:19.281987 5344 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:06:19.295310 5344 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:06:19.297514 5344 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:06:19.297676 5344 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:06:19.297888 5344 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:06:19.298229 5344 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:06:19.298276 5344 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:06:19.298300 5344 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:06:19.298323 5344 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:06:19.304250 5344 rpc_server.cc:307] RPC server started. Bound to: 127.11.254.130:46075
I20260812 21:06:19.304338 5564 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.254.130:46075 every 8 connection(s)
I20260812 21:06:19.304659 5344 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/ts-1/data/info.pb
I20260812 21:06:19.305766 5568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:19.305360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:39351 (local address 127.11.254.130:39159)
0812 21:06:19.305532 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0812 21:06:19.305622 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.190:39351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":71,"threads_started":1}
W20260812 21:06:19.306201 5565 heartbeater.cc:644] Failed to heartbeat to 127.11.254.190:39351 (0 consecutive failures): Network error: Failed to ping master at 127.11.254.190:39351: Client connection negotiation failed: client connection to 127.11.254.190:39351: connect: Connection refused (error 111)
I20260812 21:06:19.306497 5568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:19.306379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:39351 (local address 127.11.254.130:41539)
0812 21:06:19.306419 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0812 21:06:19.306477 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.190:39351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260812 21:06:19.306792 5568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:19.306716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:39351 (local address 127.11.254.130:48475)
0812 21:06:19.306751 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0812 21:06:19.306774 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.190:39351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
W20260812 21:06:19.306886 5565 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260812 21:06:19.311683 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 5344
I20260812 21:06:19.311918 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.254.131:33971
--local_ip_for_outbound_sockets=127.11.254.131
--tserver_master_addrs=127.11.254.190:39351
--webserver_port=40205
--webserver_interface=127.11.254.131
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.131.keytab
--principal=kudu/127.11.254.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.254.148:41385
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:06:19.395922 5569 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:06:19.396198 5569 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:06:19.396240 5569 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:06:19.397444 5569 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:06:19.397490 5569 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:06:19.397526 5569 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.254.131
I20260812 21:06:19.399075 5569 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.254.148:41385
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.131.keytab
--principal=kudu/127.11.254.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.254.131:33971
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.254.131
--webserver_port=40205
--tserver_master_addrs=127.11.254.190:39351
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5569
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.254.131
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:06:19.399340 5569 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:06:19.399565 5569 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:06:19.404692 5583 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:06:19.404724 5578 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:06:19.404853 5574 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:06:19.405138 5569 server_base.cc:1034] running on GCE node
I20260812 21:06:19.405337 5569 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:06:19.405583 5569 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:06:19.406726 5569 hybrid_clock.cc:648] HybridClock initialized: now 1786568779406716 us; error 32 us; skew 500 ppm
Aug 12 21:06:19 dist-test-slave-vpvm krb5kdc[3199](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568779, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:06:19.409243 5569 init.cc:377] Logged in from keytab as kudu/127.11.254.131@KRBTEST.COM (short username kudu)
I20260812 21:06:19.409693 5569 webserver.cc:460] Webserver started at http://127.11.254.131:40205/ using document root <none> and password file <none>
I20260812 21:06:19.410048 5569 fs_manager.cc:359] Metadata directory not provided
I20260812 21:06:19.410113 5569 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:06:19.411502 5569 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 21:06:19.412106 5593 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:06:19.412279 5569 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 21:06:19.412369 5569 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/ts-2/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/ts-2/wal
uuid: "bbe9664da0ab4822b18a15afc541189b"
format_stamp: "Formatted at 2026-08-12 21:06:17 on dist-test-slave-vpvm"
server_key: "fef752db8fedde413b2ee07f590d945c"
server_key_iv: "aa8415e4b63fc4e866c2a11d014c8df3"
server_key_version: "encryptionkey@0"
I20260812 21:06:19.412757 5569 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:06:19.436673 5569 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:06:19.438922 5569 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:06:19.439090 5569 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:06:19.439339 5569 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:06:19.439656 5569 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:06:19.439690 5569 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:06:19.439721 5569 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:06:19.439746 5569 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:06:19.445837 5569 rpc_server.cc:307] RPC server started. Bound to: 127.11.254.131:33971
I20260812 21:06:19.445899 5771 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.254.131:33971 every 8 connection(s)
I20260812 21:06:19.446282 5569 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786568607354921-12282-0/minicluster-data/ts-2/data/info.pb
I20260812 21:06:19.447530 5777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:19.447009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:39351 (local address 127.11.254.131:60365)
0812 21:06:19.447203 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0812 21:06:19.447302 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.190:39351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":79,"threads_started":1}
I20260812 21:06:19.447844 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 5569
W20260812 21:06:19.447985 5772 heartbeater.cc:644] Failed to heartbeat to 127.11.254.190:39351 (0 consecutive failures): Network error: Failed to ping master at 127.11.254.190:39351: Client connection negotiation failed: client connection to 127.11.254.190:39351: connect: Connection refused (error 111)
I20260812 21:06:19.448338 5777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:19.448224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:39351 (local address 127.11.254.131:40659)
0812 21:06:19.448277 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0812 21:06:19.448311 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.190:39351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260812 21:06:19.448643 5777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:19.448555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:39351 (local address 127.11.254.131:54443)
0812 21:06:19.448592 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0812 21:06:19.448619 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.190:39351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
W20260812 21:06:19.448778 5772 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260812 21:06:20.228878 5784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:20.228613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:39351 (local address 127.11.254.129:41797)
0812 21:06:20.228745 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0812 21:06:20.228809 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.190:39351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":65,"threads_started":1}
I20260812 21:06:20.307431 5785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:20.307204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:39351 (local address 127.11.254.130:42957)
0812 21:06:20.307334 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0812 21:06:20.307373 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.190:39351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":59,"threads_started":1}
I20260812 21:06:20.449438 5788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:20.449171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:39351 (local address 127.11.254.131:48071)
0812 21:06:20.449324 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0812 21:06:20.449369 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.190:39351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":78,"threads_started":1}
I20260812 21:06:20.761719 5789 raft_consensus.cc:482] T b4d484a48c5a434d9075392d3682b832 P 3ac4f761642144ab97f75d63be9843fd [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 21:06:20.761876 5789 raft_consensus.cc:504] T b4d484a48c5a434d9075392d3682b832 P 3ac4f761642144ab97f75d63be9843fd [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fb1ffccc43e4fd9b9f078c167a38d55" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 46075 } } peers { permanent_uuid: "bbe9664da0ab4822b18a15afc541189b" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 33971 } } peers { permanent_uuid: "3ac4f761642144ab97f75d63be9843fd" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 35389 } }
I20260812 21:06:20.762284 5789 leader_election.cc:288] T b4d484a48c5a434d9075392d3682b832 P 3ac4f761642144ab97f75d63be9843fd [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5fb1ffccc43e4fd9b9f078c167a38d55 (127.11.254.130:46075), bbe9664da0ab4822b18a15afc541189b (127.11.254.131:33971)
Aug 12 21:06:20 dist-test-slave-vpvm krb5kdc[3199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568779, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.129@KRBTEST.COM for kudu/127.11.254.130@KRBTEST.COM
Aug 12 21:06:20 dist-test-slave-vpvm krb5kdc[3199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568779, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.129@KRBTEST.COM for kudu/127.11.254.131@KRBTEST.COM
I20260812 21:06:20.773375 5793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:20.762644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.129:35043 (local address 127.11.254.131:33971)
0812 21:06:20.762800 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0812 21:06:20.762810 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 21:06:20.762845 (+ 35us) server_negotiation.cc:407] Connection header received
0812 21:06:20.763329 (+ 484us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:06:20.763331 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:06:20.763374 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:06:20.763507 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:06:20.764257 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:06:20.765155 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:06:20.766201 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:06:20.766624 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:06:20.766726 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:06:20.769519 (+ 2793us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:06:20.769523 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:06:20.769523 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:06:20.769536 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:06:20.771405 (+ 1869us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:06:20.772056 (+ 651us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:06:20.772058 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:06:20.772059 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:06:20.772169 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:06:20.772360 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:06:20.772361 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:06:20.772361 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:06:20.772641 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:06:20.772731 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0812 21:06:20.773103 (+ 372us) server_negotiation.cc:299] Negotiation successful
0812 21:06:20.773252 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":54,"threads_started":1}
I20260812 21:06:20.773363 5790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:20.762482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.130:46075 (local address 127.11.254.129:48851)
0812 21:06:20.762641 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0812 21:06:20.762652 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 21:06:20.762716 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:06:20.763571 (+ 855us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:06:20.763574 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:06:20.763940 (+ 366us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:06:20.764133 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:06:20.764139 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:06:20.765279 (+ 1140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:06:20.765280 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:06:20.766120 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:06:20.766123 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:06:20.766712 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:06:20.766713 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:06:20.766831 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:06:20.767364 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:06:20.767373 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:06:20.769005 (+ 1632us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:06:20.771515 (+ 2510us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:06:20.771523 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:06:20.771525 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:06:20.771751 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:06:20.772065 (+ 314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:06:20.772066 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:06:20.772066 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:06:20.772160 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:06:20.772720 (+ 560us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:06:20.772749 (+ 29us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:06:20.772920 (+ 171us) client_negotiation.cc:769] Sending connection context
0812 21:06:20.773040 (+ 120us) client_negotiation.cc:240] Negotiation successful
0812 21:06:20.773229 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"mutex_wait_us":141,"thread_start_us":87,"threads_started":1}
I20260812 21:06:20.773417 5792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:20.762530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.131:33971 (local address 127.11.254.129:35043)
0812 21:06:20.762659 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0812 21:06:20.762670 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 21:06:20.762726 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:06:20.763592 (+ 866us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:06:20.763593 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:06:20.763944 (+ 351us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:06:20.764133 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:06:20.764139 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:06:20.765259 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:06:20.765261 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:06:20.766120 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:06:20.766123 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:06:20.766714 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:06:20.766715 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:06:20.766832 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:06:20.767409 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:06:20.767414 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:06:20.769383 (+ 1969us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:06:20.771830 (+ 2447us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:06:20.771831 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:06:20.771831 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:06:20.771958 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:06:20.772248 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:06:20.772248 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:06:20.772248 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:06:20.772302 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:06:20.772720 (+ 418us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:06:20.772729 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:06:20.772920 (+ 191us) client_negotiation.cc:769] Sending connection context
0812 21:06:20.773039 (+ 119us) client_negotiation.cc:240] Negotiation successful
0812 21:06:20.773297 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"mutex_wait_us":100,"thread_start_us":69,"threads_started":1}
I20260812 21:06:20.773381 5795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:20.762656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.129:48851 (local address 127.11.254.130:46075)
0812 21:06:20.762823 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0812 21:06:20.762831 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 21:06:20.762858 (+ 27us) server_negotiation.cc:407] Connection header received
0812 21:06:20.763329 (+ 471us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:06:20.763331 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:06:20.763374 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:06:20.763489 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:06:20.764270 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:06:20.765188 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:06:20.766207 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:06:20.766621 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:06:20.766725 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:06:20.769166 (+ 2441us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:06:20.769171 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:06:20.769171 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:06:20.769185 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:06:20.771405 (+ 2220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:06:20.771850 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:06:20.771854 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:06:20.771854 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:06:20.771994 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:06:20.772218 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:06:20.772219 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:06:20.772220 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:06:20.772646 (+ 426us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:06:20.772731 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0812 21:06:20.773103 (+ 372us) server_negotiation.cc:299] Negotiation successful
0812 21:06:20.773252 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":66,"threads_started":1}
W20260812 21:06:20.774334 5158 leader_election.cc:341] T b4d484a48c5a434d9075392d3682b832 P 3ac4f761642144ab97f75d63be9843fd [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5fb1ffccc43e4fd9b9f078c167a38d55 (127.11.254.130:46075): Not found: Tablet not found: b4d484a48c5a434d9075392d3682b832
W20260812 21:06:20.774515 5154 leader_election.cc:341] T b4d484a48c5a434d9075392d3682b832 P 3ac4f761642144ab97f75d63be9843fd [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer bbe9664da0ab4822b18a15afc541189b (127.11.254.131:33971): Not found: Tablet not found: b4d484a48c5a434d9075392d3682b832
I20260812 21:06:20.774567 5154 leader_election.cc:302] T b4d484a48c5a434d9075392d3682b832 P 3ac4f761642144ab97f75d63be9843fd [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ac4f761642144ab97f75d63be9843fd; no voters: 5fb1ffccc43e4fd9b9f078c167a38d55, bbe9664da0ab4822b18a15afc541189b
I20260812 21:06:20.774669 5789 raft_consensus.cc:2726] T b4d484a48c5a434d9075392d3682b832 P 3ac4f761642144ab97f75d63be9843fd [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 21:06:21.229523 5790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:21.229375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:39351 (local address 127.11.254.129:39347)
0812 21:06:21.229437 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0812 21:06:21.229480 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.190:39351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260812 21:06:21.308135 5796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:21.307899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:39351 (local address 127.11.254.130:42925)
0812 21:06:21.308042 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0812 21:06:21.308088 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.190:39351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":67,"threads_started":1}
I20260812 21:06:21.450194 5797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:21.449931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:39351 (local address 127.11.254.131:46469)
0812 21:06:21.450086 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0812 21:06:21.450147 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.190:39351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":98,"threads_started":1}
I20260812 21:06:22.230315 5798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:22.230084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:39351 (local address 127.11.254.129:55735)
0812 21:06:22.230220 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0812 21:06:22.230266 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.190:39351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":66,"threads_started":1}
I20260812 21:06:22.287389 5799 raft_consensus.cc:482] T b4d484a48c5a434d9075392d3682b832 P 3ac4f761642144ab97f75d63be9843fd [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 21:06:22.287523 5799 raft_consensus.cc:504] T b4d484a48c5a434d9075392d3682b832 P 3ac4f761642144ab97f75d63be9843fd [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fb1ffccc43e4fd9b9f078c167a38d55" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 46075 } } peers { permanent_uuid: "bbe9664da0ab4822b18a15afc541189b" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 33971 } } peers { permanent_uuid: "3ac4f761642144ab97f75d63be9843fd" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 35389 } }
I20260812 21:06:22.287724 5799 leader_election.cc:288] T b4d484a48c5a434d9075392d3682b832 P 3ac4f761642144ab97f75d63be9843fd [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5fb1ffccc43e4fd9b9f078c167a38d55 (127.11.254.130:46075), bbe9664da0ab4822b18a15afc541189b (127.11.254.131:33971)
W20260812 21:06:22.288115 5154 leader_election.cc:341] T b4d484a48c5a434d9075392d3682b832 P 3ac4f761642144ab97f75d63be9843fd [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer bbe9664da0ab4822b18a15afc541189b (127.11.254.131:33971): Not found: Tablet not found: b4d484a48c5a434d9075392d3682b832
W20260812 21:06:22.288180 5158 leader_election.cc:341] T b4d484a48c5a434d9075392d3682b832 P 3ac4f761642144ab97f75d63be9843fd [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5fb1ffccc43e4fd9b9f078c167a38d55 (127.11.254.130:46075): Not found: Tablet not found: b4d484a48c5a434d9075392d3682b832
I20260812 21:06:22.288295 5158 leader_election.cc:302] T b4d484a48c5a434d9075392d3682b832 P 3ac4f761642144ab97f75d63be9843fd [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ac4f761642144ab97f75d63be9843fd; no voters: 5fb1ffccc43e4fd9b9f078c167a38d55, bbe9664da0ab4822b18a15afc541189b
I20260812 21:06:22.288372 5799 raft_consensus.cc:2726] T b4d484a48c5a434d9075392d3682b832 P 3ac4f761642144ab97f75d63be9843fd [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 21:06:22.308926 5800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:22.308682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:39351 (local address 127.11.254.130:52021)
0812 21:06:22.308823 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0812 21:06:22.308869 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.190:39351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":76,"threads_started":1}
I20260812 21:06:22.450901 5803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:22.450668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:39351 (local address 127.11.254.131:42607)
0812 21:06:22.450806 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0812 21:06:22.450851 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.190:39351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":69,"threads_started":1}
I20260812 21:06:23.231035 5804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:23.230755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:39351 (local address 127.11.254.129:38445)
0812 21:06:23.230922 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0812 21:06:23.230983 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.190:39351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":84,"threads_started":1}
I20260812 21:06:23.309684 5806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:23.309440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:39351 (local address 127.11.254.130:35439)
0812 21:06:23.309589 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0812 21:06:23.309635 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.190:39351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":74,"threads_started":1}
I20260812 21:06:23.451740 5808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:23.451525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:39351 (local address 127.11.254.131:49477)
0812 21:06:23.451653 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0812 21:06:23.451690 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.190:39351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":66,"threads_started":1}
I20260812 21:06:24.231839 5809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:24.231568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:39351 (local address 127.11.254.129:50091)
0812 21:06:24.231743 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0812 21:06:24.231788 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.190:39351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":86,"threads_started":1}
I20260812 21:06:24.263266 5810 raft_consensus.cc:482] T b4d484a48c5a434d9075392d3682b832 P 3ac4f761642144ab97f75d63be9843fd [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 21:06:24.263372 5810 raft_consensus.cc:504] T b4d484a48c5a434d9075392d3682b832 P 3ac4f761642144ab97f75d63be9843fd [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fb1ffccc43e4fd9b9f078c167a38d55" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 46075 } } peers { permanent_uuid: "bbe9664da0ab4822b18a15afc541189b" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 33971 } } peers { permanent_uuid: "3ac4f761642144ab97f75d63be9843fd" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 35389 } }
I20260812 21:06:24.263525 5810 leader_election.cc:288] T b4d484a48c5a434d9075392d3682b832 P 3ac4f761642144ab97f75d63be9843fd [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5fb1ffccc43e4fd9b9f078c167a38d55 (127.11.254.130:46075), bbe9664da0ab4822b18a15afc541189b (127.11.254.131:33971)
W20260812 21:06:24.263959 5154 leader_election.cc:341] T b4d484a48c5a434d9075392d3682b832 P 3ac4f761642144ab97f75d63be9843fd [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer bbe9664da0ab4822b18a15afc541189b (127.11.254.131:33971): Not found: Tablet not found: b4d484a48c5a434d9075392d3682b832
W20260812 21:06:24.264050 5158 leader_election.cc:341] T b4d484a48c5a434d9075392d3682b832 P 3ac4f761642144ab97f75d63be9843fd [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5fb1ffccc43e4fd9b9f078c167a38d55 (127.11.254.130:46075): Not found: Tablet not found: b4d484a48c5a434d9075392d3682b832
I20260812 21:06:24.264110 5158 leader_election.cc:302] T b4d484a48c5a434d9075392d3682b832 P 3ac4f761642144ab97f75d63be9843fd [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ac4f761642144ab97f75d63be9843fd; no voters: 5fb1ffccc43e4fd9b9f078c167a38d55, bbe9664da0ab4822b18a15afc541189b
I20260812 21:06:24.264206 5810 raft_consensus.cc:2726] T b4d484a48c5a434d9075392d3682b832 P 3ac4f761642144ab97f75d63be9843fd [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 21:06:24.310568 5811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:24.310245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:39351 (local address 127.11.254.130:45323)
0812 21:06:24.310459 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0812 21:06:24.310509 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.190:39351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":94,"threads_started":1}
I20260812 21:06:24.452456 5812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:24.452195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:39351 (local address 127.11.254.131:41843)
0812 21:06:24.452366 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0812 21:06:24.452406 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.190:39351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":80,"threads_started":1}
I20260812 21:06:25.232703 5813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:25.232388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:39351 (local address 127.11.254.129:40597)
0812 21:06:25.232599 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0812 21:06:25.232650 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.190:39351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":76,"threads_started":1}
I20260812 21:06:25.311457 5814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:25.311166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:39351 (local address 127.11.254.130:41275)
0812 21:06:25.311353 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0812 21:06:25.311405 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.190:39351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":83,"threads_started":1}
I20260812 21:06:25.453269 5815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:25.453009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:39351 (local address 127.11.254.131:37099)
0812 21:06:25.453170 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0812 21:06:25.453219 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.190:39351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":69,"threads_started":1}
I20260812 21:06:26.233525 5816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:26.233282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:39351 (local address 127.11.254.129:56349)
0812 21:06:26.233426 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0812 21:06:26.233474 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.190:39351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":69,"threads_started":1}
I20260812 21:06:26.312265 5817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:26.312023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:39351 (local address 127.11.254.130:59885)
0812 21:06:26.312167 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0812 21:06:26.312214 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.190:39351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":76,"threads_started":1}
I20260812 21:06:26.454128 5819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:26.453837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:39351 (local address 127.11.254.131:34319)
0812 21:06:26.454019 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0812 21:06:26.454068 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.190:39351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":93,"threads_started":1}
I20260812 21:06:27.234361 5820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:27.234128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:39351 (local address 127.11.254.129:48247)
0812 21:06:27.234262 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0812 21:06:27.234310 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.190:39351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":68,"threads_started":1}
I20260812 21:06:27.313048 5821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:27.312803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:39351 (local address 127.11.254.130:56265)
0812 21:06:27.312950 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0812 21:06:27.312998 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.190:39351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":71,"threads_started":1}
I20260812 21:06:27.454892 5823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:27.454621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:39351 (local address 127.11.254.131:38419)
0812 21:06:27.454771 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0812 21:06:27.454816 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.190:39351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":84,"threads_started":1}
I20260812 21:06:28.235116 5826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:28.234859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:39351 (local address 127.11.254.129:60327)
0812 21:06:28.235017 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0812 21:06:28.235065 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.190:39351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":86,"threads_started":1}
I20260812 21:06:28.313727 5827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:28.313519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:39351 (local address 127.11.254.130:54455)
0812 21:06:28.313651 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0812 21:06:28.313679 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.190:39351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":66,"threads_started":1}
I20260812 21:06:28.455631 5828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:28.455389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:39351 (local address 127.11.254.131:39523)
0812 21:06:28.455532 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0812 21:06:28.455581 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.190:39351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":64,"threads_started":1}
I20260812 21:06:28.861361 5830 raft_consensus.cc:482] T b4d484a48c5a434d9075392d3682b832 P 3ac4f761642144ab97f75d63be9843fd [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 21:06:28.861462 5830 raft_consensus.cc:504] T b4d484a48c5a434d9075392d3682b832 P 3ac4f761642144ab97f75d63be9843fd [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fb1ffccc43e4fd9b9f078c167a38d55" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 46075 } } peers { permanent_uuid: "bbe9664da0ab4822b18a15afc541189b" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 33971 } } peers { permanent_uuid: "3ac4f761642144ab97f75d63be9843fd" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 35389 } }
I20260812 21:06:28.861656 5830 leader_election.cc:288] T b4d484a48c5a434d9075392d3682b832 P 3ac4f761642144ab97f75d63be9843fd [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5fb1ffccc43e4fd9b9f078c167a38d55 (127.11.254.130:46075), bbe9664da0ab4822b18a15afc541189b (127.11.254.131:33971)
W20260812 21:06:28.862071 5154 leader_election.cc:341] T b4d484a48c5a434d9075392d3682b832 P 3ac4f761642144ab97f75d63be9843fd [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer bbe9664da0ab4822b18a15afc541189b (127.11.254.131:33971): Not found: Tablet not found: b4d484a48c5a434d9075392d3682b832
W20260812 21:06:28.862169 5158 leader_election.cc:341] T b4d484a48c5a434d9075392d3682b832 P 3ac4f761642144ab97f75d63be9843fd [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5fb1ffccc43e4fd9b9f078c167a38d55 (127.11.254.130:46075): Not found: Tablet not found: b4d484a48c5a434d9075392d3682b832
I20260812 21:06:28.862232 5158 leader_election.cc:302] T b4d484a48c5a434d9075392d3682b832 P 3ac4f761642144ab97f75d63be9843fd [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ac4f761642144ab97f75d63be9843fd; no voters: 5fb1ffccc43e4fd9b9f078c167a38d55, bbe9664da0ab4822b18a15afc541189b
I20260812 21:06:28.862298 5830 raft_consensus.cc:2726] T b4d484a48c5a434d9075392d3682b832 P 3ac4f761642144ab97f75d63be9843fd [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 21:06:29.235925 5832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:29.235648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:39351 (local address 127.11.254.129:35333)
0812 21:06:29.235819 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0812 21:06:29.235867 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.190:39351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":72,"threads_started":1}
I20260812 21:06:29.314536 5833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:29.314277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:39351 (local address 127.11.254.130:57095)
0812 21:06:29.314417 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0812 21:06:29.314466 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.190:39351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":68,"threads_started":1}
I20260812 21:06:29.456279 5834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:29.456058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:39351 (local address 127.11.254.131:36607)
0812 21:06:29.456193 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0812 21:06:29.456230 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.190:39351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":65,"threads_started":1}
I20260812 21:06:30.236848 5835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:30.236568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:39351 (local address 127.11.254.129:55359)
0812 21:06:30.236747 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0812 21:06:30.236796 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.190:39351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":72,"threads_started":1}
I20260812 21:06:30.315469 5836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:30.315175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:39351 (local address 127.11.254.130:49475)
0812 21:06:30.315356 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0812 21:06:30.315416 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.190:39351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":79,"threads_started":1}
I20260812 21:06:30.457064 5837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:30.456783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:39351 (local address 127.11.254.131:39469)
0812 21:06:30.456969 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0812 21:06:30.457013 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.190:39351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":94,"threads_started":1}
I20260812 21:06:31.237736 5839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:31.237442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:39351 (local address 127.11.254.129:49095)
0812 21:06:31.237634 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0812 21:06:31.237683 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.190:39351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":96,"threads_started":1}
I20260812 21:06:31.316262 5841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:31.316011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:39351 (local address 127.11.254.130:51379)
0812 21:06:31.316168 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0812 21:06:31.316212 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.190:39351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":69,"threads_started":1}
I20260812 21:06:31.457850 5842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:31.457587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:39351 (local address 127.11.254.131:49407)
0812 21:06:31.457750 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0812 21:06:31.457796 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.190:39351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":66,"threads_started":1}
I20260812 21:06:32.238551 5843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:32.238295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:39351 (local address 127.11.254.129:52175)
0812 21:06:32.238449 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0812 21:06:32.238498 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.190:39351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":74,"threads_started":1}
I20260812 21:06:32.316998 5844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:32.316759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:39351 (local address 127.11.254.130:33973)
0812 21:06:32.316899 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0812 21:06:32.316945 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.190:39351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":66,"threads_started":1}
I20260812 21:06:32.458645 5846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:32.458389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:39351 (local address 127.11.254.131:43775)
0812 21:06:32.458543 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0812 21:06:32.458594 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.190:39351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":70,"threads_started":1}
I20260812 21:06:33.099521 5850 raft_consensus.cc:482] T b4d484a48c5a434d9075392d3682b832 P 3ac4f761642144ab97f75d63be9843fd [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 21:06:33.099634 5850 raft_consensus.cc:504] T b4d484a48c5a434d9075392d3682b832 P 3ac4f761642144ab97f75d63be9843fd [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fb1ffccc43e4fd9b9f078c167a38d55" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 46075 } } peers { permanent_uuid: "bbe9664da0ab4822b18a15afc541189b" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 33971 } } peers { permanent_uuid: "3ac4f761642144ab97f75d63be9843fd" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 35389 } }
I20260812 21:06:33.099807 5850 leader_election.cc:288] T b4d484a48c5a434d9075392d3682b832 P 3ac4f761642144ab97f75d63be9843fd [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5fb1ffccc43e4fd9b9f078c167a38d55 (127.11.254.130:46075), bbe9664da0ab4822b18a15afc541189b (127.11.254.131:33971)
W20260812 21:06:33.100369 5158 leader_election.cc:341] T b4d484a48c5a434d9075392d3682b832 P 3ac4f761642144ab97f75d63be9843fd [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5fb1ffccc43e4fd9b9f078c167a38d55 (127.11.254.130:46075): Not found: Tablet not found: b4d484a48c5a434d9075392d3682b832
W20260812 21:06:33.100476 5154 leader_election.cc:341] T b4d484a48c5a434d9075392d3682b832 P 3ac4f761642144ab97f75d63be9843fd [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer bbe9664da0ab4822b18a15afc541189b (127.11.254.131:33971): Not found: Tablet not found: b4d484a48c5a434d9075392d3682b832
I20260812 21:06:33.100521 5154 leader_election.cc:302] T b4d484a48c5a434d9075392d3682b832 P 3ac4f761642144ab97f75d63be9843fd [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ac4f761642144ab97f75d63be9843fd; no voters: 5fb1ffccc43e4fd9b9f078c167a38d55, bbe9664da0ab4822b18a15afc541189b
I20260812 21:06:33.100633 5850 raft_consensus.cc:2726] T b4d484a48c5a434d9075392d3682b832 P 3ac4f761642144ab97f75d63be9843fd [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 21:06:33.239439 5851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:33.239185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:39351 (local address 127.11.254.129:37575)
0812 21:06:33.239334 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0812 21:06:33.239386 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.190:39351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":71,"threads_started":1}
I20260812 21:06:33.317786 5853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:33.317529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:39351 (local address 127.11.254.130:47271)
0812 21:06:33.317666 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0812 21:06:33.317716 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.190:39351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":73,"threads_started":1}
I20260812 21:06:33.459381 5854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:33.459155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:39351 (local address 127.11.254.131:50487)
0812 21:06:33.459284 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0812 21:06:33.459330 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.190:39351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":63,"threads_started":1}
I20260812 21:06:34.240341 5855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:34.240117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:39351 (local address 127.11.254.129:49693)
0812 21:06:34.240248 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0812 21:06:34.240293 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.190:39351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":72,"threads_started":1}
I20260812 21:06:34.318590 5856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:34.318346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:39351 (local address 127.11.254.130:54821)
0812 21:06:34.318498 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0812 21:06:34.318543 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.190:39351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":70,"threads_started":1}
I20260812 21:06:34.453891 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 5084
I20260812 21:06:34.463416 5857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:34.461895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:39351 (local address 127.11.254.131:58903)
0812 21:06:34.463293 (+ 1398us) negotiation.cc:107] Waiting for socket to connect
0812 21:06:34.463346 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.190:39351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1375,"thread_start_us":1285,"threads_started":1}
I20260812 21:06:34.476801 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 5344
I20260812 21:06:34.490494 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 5569
2026-08-12T21:06:34Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (18693 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 21:06:34 dist-test-slave-vpvm krb5kdc[5864](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 21:06:34 dist-test-slave-vpvm krb5kdc[5864](info): set up 2 sockets
Aug 12 21:06:34 dist-test-slave-vpvm krb5kdc[5864](info): commencing operation
krb5kdc: starting...
W20260812 21:06:35.694600 12282 mini_kdc.cc:121] Time spent starting KDC: real 1.172s user 0.006s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/krb5kdc/test-user.keytab.
Aug 12 21:06:35 dist-test-slave-vpvm krb5kdc[5864](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568795, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T21:06:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T21:06:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.254.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.254.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.254.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.190.keytab.
Entry for principal kudu/127.11.254.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.190.keytab.
I20260812 21:06:35.825358 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.254.190:38123
--webserver_interface=127.11.254.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.190.keytab
--principal=kudu/127.11.254.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.254.148:34711
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.254.190:38123
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:06:35.907882 5897 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:06:35.908102 5897 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:06:35.908133 5897 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:06:35.909329 5897 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260812 21:06:35.909374 5897 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260812 21:06:35.909390 5897 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260812 21:06:35.909404 5897 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 21:06:35.909421 5897 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:06:35.909437 5897 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:06:35.909452 5897 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 21:06:35.909471 5897 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 21:06:35.911070 5897 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.254.148:34711
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.11.254.190:38123
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.190.keytab
--principal=kudu/127.11.254.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.254.190:38123
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.254.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5897
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:06:35.911341 5897 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:06:35.911576 5897 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:06:35.917425 5908 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:06:35.917456 5897 server_base.cc:1034] running on GCE node
W20260812 21:06:35.917426 5906 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:06:35.917426 5905 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:06:35.917920 5897 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:06:35.918285 5897 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:06:35.919435 5897 hybrid_clock.cc:648] HybridClock initialized: now 1786568795919411 us; error 44 us; skew 500 ppm
Aug 12 21:06:35 dist-test-slave-vpvm krb5kdc[5864](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568795, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:06:35.922065 5897 init.cc:377] Logged in from keytab as kudu/127.11.254.190@KRBTEST.COM (short username kudu)
I20260812 21:06:35.922538 5897 webserver.cc:460] Webserver started at http://127.11.254.190:33283/ using document root <none> and password file <none>
I20260812 21:06:35.922818 5897 fs_manager.cc:359] Metadata directory not provided
I20260812 21:06:35.922865 5897 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:06:35.922995 5897 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:06:35.924101 5897 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/master-0/data/instance:
uuid: "36a27cabe4ae44568b7adc99d86a1430"
format_stamp: "Formatted at 2026-08-12 21:06:35 on dist-test-slave-vpvm"
server_key: "0c538b3805728f70b6f07f44ee6460b6"
server_key_iv: "5f1b651e34f771fe03d7352717eea675"
server_key_version: "encryptionkey@0"
I20260812 21:06:35.924448 5897 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/master-0/wal/instance:
uuid: "36a27cabe4ae44568b7adc99d86a1430"
format_stamp: "Formatted at 2026-08-12 21:06:35 on dist-test-slave-vpvm"
server_key: "0c538b3805728f70b6f07f44ee6460b6"
server_key_iv: "5f1b651e34f771fe03d7352717eea675"
server_key_version: "encryptionkey@0"
I20260812 21:06:35.925808 5897 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 21:06:35.926667 5919 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:06:35.926841 5897 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 21:06:35.926927 5897 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/master-0/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/master-0/wal
uuid: "36a27cabe4ae44568b7adc99d86a1430"
format_stamp: "Formatted at 2026-08-12 21:06:35 on dist-test-slave-vpvm"
server_key: "0c538b3805728f70b6f07f44ee6460b6"
server_key_iv: "5f1b651e34f771fe03d7352717eea675"
server_key_version: "encryptionkey@0"
I20260812 21:06:35.927003 5897 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:06:35.944653 5897 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:06:35.946696 5897 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:06:35.946830 5897 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:06:35.951151 5897 rpc_server.cc:307] RPC server started. Bound to: 127.11.254.190:38123
I20260812 21:06:35.951205 6002 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.254.190:38123 every 8 connection(s)
I20260812 21:06:35.951545 5897 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
I20260812 21:06:35.952498 6005 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:06:35.954864 6005 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36a27cabe4ae44568b7adc99d86a1430: Bootstrap starting.
I20260812 21:06:35.955464 6005 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 36a27cabe4ae44568b7adc99d86a1430: Neither blocks nor log segments found. Creating new log.
I20260812 21:06:35.955755 6005 log.cc:824] T 00000000000000000000000000000000 P 36a27cabe4ae44568b7adc99d86a1430: Log is configured to *not* fsync() on all Append() calls
I20260812 21:06:35.956466 6005 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36a27cabe4ae44568b7adc99d86a1430: No bootstrap required, opened a new log
I20260812 21:06:35.958200 6005 raft_consensus.cc:348] T 00000000000000000000000000000000 P 36a27cabe4ae44568b7adc99d86a1430 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36a27cabe4ae44568b7adc99d86a1430" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 38123 } }
I20260812 21:06:35.958348 6005 raft_consensus.cc:374] T 00000000000000000000000000000000 P 36a27cabe4ae44568b7adc99d86a1430 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:06:35.958370 6005 raft_consensus.cc:729] T 00000000000000000000000000000000 P 36a27cabe4ae44568b7adc99d86a1430 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36a27cabe4ae44568b7adc99d86a1430, State: Initialized, Role: FOLLOWER
I20260812 21:06:35.958513 6005 consensus_queue.cc:260] T 00000000000000000000000000000000 P 36a27cabe4ae44568b7adc99d86a1430 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36a27cabe4ae44568b7adc99d86a1430" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 38123 } }
I20260812 21:06:35.958586 6005 raft_consensus.cc:388] T 00000000000000000000000000000000 P 36a27cabe4ae44568b7adc99d86a1430 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 21:06:35.958609 6005 raft_consensus.cc:482] T 00000000000000000000000000000000 P 36a27cabe4ae44568b7adc99d86a1430 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 21:06:35.958657 6005 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 36a27cabe4ae44568b7adc99d86a1430 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:06:35.959278 6005 raft_consensus.cc:504] T 00000000000000000000000000000000 P 36a27cabe4ae44568b7adc99d86a1430 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36a27cabe4ae44568b7adc99d86a1430" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 38123 } }
I20260812 21:06:35.959375 6005 leader_election.cc:302] T 00000000000000000000000000000000 P 36a27cabe4ae44568b7adc99d86a1430 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 36a27cabe4ae44568b7adc99d86a1430; no voters:
I20260812 21:06:35.959570 6005 leader_election.cc:288] T 00000000000000000000000000000000 P 36a27cabe4ae44568b7adc99d86a1430 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 21:06:35.959616 6010 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 36a27cabe4ae44568b7adc99d86a1430 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:06:35.959765 6010 raft_consensus.cc:686] T 00000000000000000000000000000000 P 36a27cabe4ae44568b7adc99d86a1430 [term 1 LEADER]: Becoming Leader. State: Replica: 36a27cabe4ae44568b7adc99d86a1430, State: Running, Role: LEADER
I20260812 21:06:35.959830 6005 sys_catalog.cc:564] T 00000000000000000000000000000000 P 36a27cabe4ae44568b7adc99d86a1430 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 21:06:35.959864 6010 consensus_queue.cc:237] T 00000000000000000000000000000000 P 36a27cabe4ae44568b7adc99d86a1430 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36a27cabe4ae44568b7adc99d86a1430" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 38123 } }
I20260812 21:06:35.960538 6012 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36a27cabe4ae44568b7adc99d86a1430 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 36a27cabe4ae44568b7adc99d86a1430. Latest consensus state: current_term: 1 leader_uuid: "36a27cabe4ae44568b7adc99d86a1430" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36a27cabe4ae44568b7adc99d86a1430" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 38123 } } }
I20260812 21:06:35.960638 6012 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36a27cabe4ae44568b7adc99d86a1430 [sys.catalog]: This master's current role is: LEADER
I20260812 21:06:35.960517 6011 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36a27cabe4ae44568b7adc99d86a1430 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "36a27cabe4ae44568b7adc99d86a1430" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36a27cabe4ae44568b7adc99d86a1430" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 38123 } } }
I20260812 21:06:35.960924 6011 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36a27cabe4ae44568b7adc99d86a1430 [sys.catalog]: This master's current role is: LEADER
I20260812 21:06:35.961105 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 5897
I20260812 21:06:35.961197 6024 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 21:06:35.961256 12282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/master-0/wal/instance
I20260812 21:06:35.961514 12282 external_mini_cluster.cc:1428] Setting key 2679a1122f58a55a9cda556ec44e4a9c
I20260812 21:06:35.961779 6024 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 21:06:35.963652 6024 catalog_manager.cc:1347] Generated new cluster ID: 27a149e698fb48b0a6d33906f55215fa
I20260812 21:06:35.963706 6024 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 21:06:35 dist-test-slave-vpvm krb5kdc[5864](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568795, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.254.190@KRBTEST.COM
I20260812 21:06:35.969689 6024 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 21:06:35.970307 6024 catalog_manager.cc:1504] Loading token signing keys...
I20260812 21:06:35.973826 6040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:35.962105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55790 (local address 127.11.254.190:38123)
0812 21:06:35.962279 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0812 21:06:35.962289 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 21:06:35.962499 (+ 210us) server_negotiation.cc:407] Connection header received
0812 21:06:35.962631 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:06:35.962638 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:06:35.962697 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:06:35.962835 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:06:35.963643 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:06:35.965043 (+ 1400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:06:35.966197 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:06:35.966822 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:06:35.966928 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:06:35.969931 (+ 3003us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:06:35.969936 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:06:35.969936 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:06:35.969972 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:06:35.972043 (+ 2071us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:06:35.972470 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:06:35.972473 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:06:35.972473 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:06:35.972591 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:06:35.972790 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:06:35.972791 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:06:35.972792 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:06:35.973168 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:06:35.973334 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0812 21:06:35.973408 (+ 74us) server_negotiation.cc:299] Negotiation successful
0812 21:06:35.973577 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":74,"threads_started":1}
I20260812 21:06:35.976121 6024 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 36a27cabe4ae44568b7adc99d86a1430: Generated new TSK 0
I20260812 21:06:35.976271 6024 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.254.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.254.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.254.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.129.keytab.
Entry for principal kudu/127.11.254.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.129.keytab.
I20260812 21:06:36.013795 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.254.129:0
--local_ip_for_outbound_sockets=127.11.254.129
--webserver_interface=127.11.254.129
--webserver_port=0
--tserver_master_addrs=127.11.254.190:38123
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.129.keytab
--principal=kudu/127.11.254.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.254.148:34711
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:06:36.100417 6047 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:06:36.100646 6047 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:06:36.100680 6047 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:06:36.101893 6047 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:06:36.101979 6047 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:06:36.102020 6047 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.254.129
I20260812 21:06:36.103637 6047 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.254.148:34711
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.129.keytab
--principal=kudu/127.11.254.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.254.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.254.129
--webserver_port=0
--tserver_master_addrs=127.11.254.190:38123
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6047
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.254.129
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:06:36.103917 6047 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:06:36.104169 6047 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:06:36.109479 6055 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:06:36.109483 6057 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:06:36.109525 6047 server_base.cc:1034] running on GCE node
W20260812 21:06:36.109479 6060 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:06:36.109928 6047 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:06:36.110244 6047 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:06:36.111419 6047 hybrid_clock.cc:648] HybridClock initialized: now 1786568796111403 us; error 33 us; skew 500 ppm
Aug 12 21:06:36 dist-test-slave-vpvm krb5kdc[5864](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568796, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:06:36.114324 6047 init.cc:377] Logged in from keytab as kudu/127.11.254.129@KRBTEST.COM (short username kudu)
I20260812 21:06:36.114825 6047 webserver.cc:460] Webserver started at http://127.11.254.129:34443/ using document root <none> and password file <none>
I20260812 21:06:36.115083 6047 fs_manager.cc:359] Metadata directory not provided
I20260812 21:06:36.115146 6047 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:06:36.115290 6047 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:06:36.116392 6047 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/ts-0/data/instance:
uuid: "813f1c245a4e49c499d584012bf01a52"
format_stamp: "Formatted at 2026-08-12 21:06:36 on dist-test-slave-vpvm"
server_key: "a606426d65f79390689a46a5466b23a5"
server_key_iv: "efa852fc71081e51e3bd8973ee227f20"
server_key_version: "encryptionkey@0"
I20260812 21:06:36.116768 6047 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/ts-0/wal/instance:
uuid: "813f1c245a4e49c499d584012bf01a52"
format_stamp: "Formatted at 2026-08-12 21:06:36 on dist-test-slave-vpvm"
server_key: "a606426d65f79390689a46a5466b23a5"
server_key_iv: "efa852fc71081e51e3bd8973ee227f20"
server_key_version: "encryptionkey@0"
I20260812 21:06:36.118274 6047 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.001s
I20260812 21:06:36.119131 6069 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:06:36.119293 6047 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 21:06:36.119387 6047 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/ts-0/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/ts-0/wal
uuid: "813f1c245a4e49c499d584012bf01a52"
format_stamp: "Formatted at 2026-08-12 21:06:36 on dist-test-slave-vpvm"
server_key: "a606426d65f79390689a46a5466b23a5"
server_key_iv: "efa852fc71081e51e3bd8973ee227f20"
server_key_version: "encryptionkey@0"
I20260812 21:06:36.119458 6047 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:06:36.132247 6047 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:06:36.134274 6047 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:06:36.134430 6047 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:06:36.134682 6047 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:06:36.134995 6047 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:06:36.135031 6047 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:06:36.135063 6047 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:06:36.135080 6047 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:06:36.141008 6047 rpc_server.cc:307] RPC server started. Bound to: 127.11.254.129:38685
I20260812 21:06:36.141074 6240 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.254.129:38685 every 8 connection(s)
I20260812 21:06:36.141419 6047 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/ts-0/data/info.pb
Aug 12 21:06:36 dist-test-slave-vpvm krb5kdc[5864](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568796, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.129@KRBTEST.COM for kudu/127.11.254.190@KRBTEST.COM
I20260812 21:06:36.150251 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 6047
I20260812 21:06:36.150359 12282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/ts-0/wal/instance
I20260812 21:06:36.150650 12282 external_mini_cluster.cc:1428] Setting key 8c2c68474fddb9ba42b06c8f6c41098f
I20260812 21:06:36.151558 6040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:36.142210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.129:50439 (local address 127.11.254.190:38123)
0812 21:06:36.142288 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0812 21:06:36.142290 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:06:36.142479 (+ 189us) server_negotiation.cc:407] Connection header received
0812 21:06:36.143069 (+ 590us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:06:36.143071 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:06:36.143093 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:06:36.143163 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:06:36.144202 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:06:36.144738 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:06:36.145787 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:06:36.146058 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:06:36.146129 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:06:36.148364 (+ 2235us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:06:36.148366 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:06:36.148366 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:06:36.148374 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:06:36.149705 (+ 1331us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:06:36.150268 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:06:36.150269 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:06:36.150269 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:06:36.150329 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:06:36.150747 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:06:36.150750 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:06:36.150750 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:06:36.150913 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:06:36.150958 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0812 21:06:36.151362 (+ 404us) server_negotiation.cc:299] Negotiation successful
0812 21:06:36.151486 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260812 21:06:36.151753 6244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:36.142159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:38123 (local address 127.11.254.129:50439)
0812 21:06:36.142349 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0812 21:06:36.142389 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 21:06:36.142941 (+ 552us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:06:36.143249 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:06:36.143253 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:06:36.143864 (+ 611us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:06:36.144089 (+ 225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:06:36.144094 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:06:36.144851 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:06:36.144852 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:06:36.145688 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:06:36.145691 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:06:36.146140 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:06:36.146142 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:06:36.146276 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:06:36.146798 (+ 522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:06:36.146808 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:06:36.148271 (+ 1463us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:06:36.149808 (+ 1537us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:06:36.149816 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:06:36.149819 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:06:36.150189 (+ 370us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:06:36.150406 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:06:36.150407 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:06:36.150408 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:06:36.150588 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:06:36.151011 (+ 423us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:06:36.151019 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:06:36.151204 (+ 185us) client_negotiation.cc:769] Sending connection context
0812 21:06:36.151313 (+ 109us) client_negotiation.cc:240] Negotiation successful
0812 21:06:36.151502 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":94,"threads_started":1}
I20260812 21:06:36.152262 6241 heartbeater.cc:347] Connected to a master server at 127.11.254.190:38123
I20260812 21:06:36.152374 6241 heartbeater.cc:464] Registering TS with master...
I20260812 21:06:36.152580 6241 heartbeater.cc:511] Master 127.11.254.190:38123 requested a full tablet report, sending...
I20260812 21:06:36.153129 5943 ts_manager.cc:194] Registered new tserver with Master: 813f1c245a4e49c499d584012bf01a52 (127.11.254.129:38685)
I20260812 21:06:36.154122 5943 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.254.129@KRBTEST.COM'} at 127.11.254.129:50439
WARNING: no policy specified for kudu/127.11.254.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.254.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.254.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.130.keytab.
Entry for principal kudu/127.11.254.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.130.keytab.
I20260812 21:06:36.180440 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.254.130:0
--local_ip_for_outbound_sockets=127.11.254.130
--webserver_interface=127.11.254.130
--webserver_port=0
--tserver_master_addrs=127.11.254.190:38123
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.130.keytab
--principal=kudu/127.11.254.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.254.148:34711
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:06:36.268231 6247 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:06:36.268460 6247 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:06:36.268486 6247 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:06:36.269716 6247 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:06:36.269765 6247 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:06:36.269793 6247 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.254.130
I20260812 21:06:36.271420 6247 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.254.148:34711
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.130.keytab
--principal=kudu/127.11.254.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.254.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.254.130
--webserver_port=0
--tserver_master_addrs=127.11.254.190:38123
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6247
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.254.130
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:06:36.271675 6247 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:06:36.271932 6247 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:06:36.277204 6254 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:06:36.277381 6247 server_base.cc:1034] running on GCE node
W20260812 21:06:36.277237 6258 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:06:36.277292 6253 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:06:36.277716 6247 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:06:36.278002 6247 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:06:36.279167 6247 hybrid_clock.cc:648] HybridClock initialized: now 1786568796279141 us; error 39 us; skew 500 ppm
Aug 12 21:06:36 dist-test-slave-vpvm krb5kdc[5864](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568796, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:06:36.281747 6247 init.cc:377] Logged in from keytab as kudu/127.11.254.130@KRBTEST.COM (short username kudu)
I20260812 21:06:36.282284 6247 webserver.cc:460] Webserver started at http://127.11.254.130:43347/ using document root <none> and password file <none>
I20260812 21:06:36.282537 6247 fs_manager.cc:359] Metadata directory not provided
I20260812 21:06:36.282590 6247 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:06:36.282743 6247 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:06:36.283864 6247 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/ts-1/data/instance:
uuid: "8eb6c52ef06a4d5f97762c59705d0047"
format_stamp: "Formatted at 2026-08-12 21:06:36 on dist-test-slave-vpvm"
server_key: "ec49e5db1cd29defa3404affc9b1cc98"
server_key_iv: "bc96b50eff0bf064cfb099c763c04d06"
server_key_version: "encryptionkey@0"
I20260812 21:06:36.284229 6247 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/ts-1/wal/instance:
uuid: "8eb6c52ef06a4d5f97762c59705d0047"
format_stamp: "Formatted at 2026-08-12 21:06:36 on dist-test-slave-vpvm"
server_key: "ec49e5db1cd29defa3404affc9b1cc98"
server_key_iv: "bc96b50eff0bf064cfb099c763c04d06"
server_key_version: "encryptionkey@0"
I20260812 21:06:36.285651 6247 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260812 21:06:36.286525 6268 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:06:36.286743 6247 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 21:06:36.286835 6247 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/ts-1/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/ts-1/wal
uuid: "8eb6c52ef06a4d5f97762c59705d0047"
format_stamp: "Formatted at 2026-08-12 21:06:36 on dist-test-slave-vpvm"
server_key: "ec49e5db1cd29defa3404affc9b1cc98"
server_key_iv: "bc96b50eff0bf064cfb099c763c04d06"
server_key_version: "encryptionkey@0"
I20260812 21:06:36.286912 6247 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:06:36.314708 6247 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:06:36.317245 6247 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:06:36.317423 6247 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:06:36.317713 6247 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:06:36.318096 6247 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:06:36.318140 6247 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:06:36.318172 6247 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:06:36.318197 6247 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:06:36.324318 6247 rpc_server.cc:307] RPC server started. Bound to: 127.11.254.130:39975
I20260812 21:06:36.324399 6459 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.254.130:39975 every 8 connection(s)
I20260812 21:06:36.324739 6247 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/ts-1/data/info.pb
I20260812 21:06:36.326840 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 6247
I20260812 21:06:36.326933 12282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/ts-1/wal/instance
I20260812 21:06:36.327124 12282 external_mini_cluster.cc:1428] Setting key c663cff136f8b7c5896a60d5e39be6b2
Aug 12 21:06:36 dist-test-slave-vpvm krb5kdc[5864](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568796, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.130@KRBTEST.COM for kudu/127.11.254.190@KRBTEST.COM
I20260812 21:06:36.335491 6040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:36.325559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.130:45013 (local address 127.11.254.190:38123)
0812 21:06:36.325631 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0812 21:06:36.325634 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 21:06:36.325909 (+ 275us) server_negotiation.cc:407] Connection header received
0812 21:06:36.326493 (+ 584us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:06:36.326495 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:06:36.326507 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:06:36.326573 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:06:36.327635 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:06:36.328242 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:06:36.329307 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:06:36.329601 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:06:36.329666 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:06:36.332232 (+ 2566us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:06:36.332235 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:06:36.332235 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:06:36.332245 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:06:36.333916 (+ 1671us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:06:36.334420 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:06:36.334422 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:06:36.334422 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:06:36.334507 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:06:36.334760 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:06:36.334761 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:06:36.334761 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:06:36.334921 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:06:36.334993 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0812 21:06:36.335280 (+ 287us) server_negotiation.cc:299] Negotiation successful
0812 21:06:36.335379 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260812 21:06:36.335565 6465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:36.325585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:38123 (local address 127.11.254.130:45013)
0812 21:06:36.325778 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0812 21:06:36.325819 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 21:06:36.326360 (+ 541us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:06:36.326671 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:06:36.326674 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:06:36.327326 (+ 652us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:06:36.327547 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:06:36.327552 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:06:36.328350 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:06:36.328351 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:06:36.329208 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:06:36.329211 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:06:36.329715 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:06:36.329716 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:06:36.329849 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:06:36.330455 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:06:36.330465 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:06:36.332098 (+ 1633us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:06:36.334052 (+ 1954us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:06:36.334063 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:06:36.334065 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:06:36.334331 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:06:36.334574 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:06:36.334574 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:06:36.334575 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:06:36.334694 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:06:36.334983 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:06:36.334986 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:06:36.335135 (+ 149us) client_negotiation.cc:769] Sending connection context
0812 21:06:36.335222 (+ 87us) client_negotiation.cc:240] Negotiation successful
0812 21:06:36.335370 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":95,"threads_started":1}
I20260812 21:06:36.336170 6460 heartbeater.cc:347] Connected to a master server at 127.11.254.190:38123
I20260812 21:06:36.336288 6460 heartbeater.cc:464] Registering TS with master...
I20260812 21:06:36.336498 6460 heartbeater.cc:511] Master 127.11.254.190:38123 requested a full tablet report, sending...
I20260812 21:06:36.337039 5943 ts_manager.cc:194] Registered new tserver with Master: 8eb6c52ef06a4d5f97762c59705d0047 (127.11.254.130:39975)
WARNING: no policy specified for kudu/127.11.254.131@KRBTEST.COM; defaulting to no policy
I20260812 21:06:36.337561 5943 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.254.130@KRBTEST.COM'} at 127.11.254.130:45013
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.254.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.254.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.131.keytab.
Entry for principal kudu/127.11.254.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.131.keytab.
I20260812 21:06:36.356922 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.254.131:0
--local_ip_for_outbound_sockets=127.11.254.131
--webserver_interface=127.11.254.131
--webserver_port=0
--tserver_master_addrs=127.11.254.190:38123
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.131.keytab
--principal=kudu/127.11.254.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.254.148:34711
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:06:36.442137 6468 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:06:36.442354 6468 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:06:36.442378 6468 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:06:36.443593 6468 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:06:36.443639 6468 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:06:36.443666 6468 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.254.131
I20260812 21:06:36.445147 6468 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.254.148:34711
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.131.keytab
--principal=kudu/127.11.254.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.254.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.254.131
--webserver_port=0
--tserver_master_addrs=127.11.254.190:38123
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6468
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.254.131
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:06:36.445350 6468 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:06:36.445595 6468 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:06:36.450903 6477 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:06:36.450903 6482 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:06:36.451052 6468 server_base.cc:1034] running on GCE node
W20260812 21:06:36.450917 6479 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:06:36.451498 6468 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:06:36.451749 6468 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:06:36.452913 6468 hybrid_clock.cc:648] HybridClock initialized: now 1786568796452899 us; error 30 us; skew 500 ppm
Aug 12 21:06:36 dist-test-slave-vpvm krb5kdc[5864](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568796, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:06:36.455478 6468 init.cc:377] Logged in from keytab as kudu/127.11.254.131@KRBTEST.COM (short username kudu)
I20260812 21:06:36.455884 6468 webserver.cc:460] Webserver started at http://127.11.254.131:46091/ using document root <none> and password file <none>
I20260812 21:06:36.456122 6468 fs_manager.cc:359] Metadata directory not provided
I20260812 21:06:36.456168 6468 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:06:36.456285 6468 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:06:36.457348 6468 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/ts-2/data/instance:
uuid: "024bc23e991c46bc813212a50b07f065"
format_stamp: "Formatted at 2026-08-12 21:06:36 on dist-test-slave-vpvm"
server_key: "7a0d4318ad00daca63381859cea667cd"
server_key_iv: "319262ae1c58b20aebe38eb1be4a839d"
server_key_version: "encryptionkey@0"
I20260812 21:06:36.457666 6468 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/ts-2/wal/instance:
uuid: "024bc23e991c46bc813212a50b07f065"
format_stamp: "Formatted at 2026-08-12 21:06:36 on dist-test-slave-vpvm"
server_key: "7a0d4318ad00daca63381859cea667cd"
server_key_iv: "319262ae1c58b20aebe38eb1be4a839d"
server_key_version: "encryptionkey@0"
I20260812 21:06:36.458995 6468 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 21:06:36.459707 6493 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:06:36.459887 6468 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 21:06:36.459971 6468 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/ts-2/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/ts-2/wal
uuid: "024bc23e991c46bc813212a50b07f065"
format_stamp: "Formatted at 2026-08-12 21:06:36 on dist-test-slave-vpvm"
server_key: "7a0d4318ad00daca63381859cea667cd"
server_key_iv: "319262ae1c58b20aebe38eb1be4a839d"
server_key_version: "encryptionkey@0"
I20260812 21:06:36.460050 6468 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:06:36.479415 6468 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:06:36.481714 6468 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:06:36.481838 6468 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:06:36.482084 6468 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:06:36.482398 6468 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:06:36.482429 6468 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:06:36.482458 6468 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:06:36.482475 6468 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:06:36.488102 6468 rpc_server.cc:307] RPC server started. Bound to: 127.11.254.131:38309
I20260812 21:06:36.488206 6668 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.254.131:38309 every 8 connection(s)
I20260812 21:06:36.488494 6468 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/ts-2/data/info.pb
I20260812 21:06:36.493361 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 6468
I20260812 21:06:36.493461 12282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/ts-2/wal/instance
I20260812 21:06:36.493642 12282 external_mini_cluster.cc:1428] Setting key 50276932872af0e049123273e48c4de7
Aug 12 21:06:36 dist-test-slave-vpvm krb5kdc[5864](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568796, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.131@KRBTEST.COM for kudu/127.11.254.190@KRBTEST.COM
I20260812 21:06:36.497928 6040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:36.489272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.131:58323 (local address 127.11.254.190:38123)
0812 21:06:36.489336 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0812 21:06:36.489338 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:06:36.489595 (+ 257us) server_negotiation.cc:407] Connection header received
0812 21:06:36.490177 (+ 582us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:06:36.490178 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:06:36.490189 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:06:36.490249 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:06:36.491282 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:06:36.491809 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:06:36.492856 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:06:36.493038 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:06:36.493083 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:06:36.495492 (+ 2409us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:06:36.495493 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:06:36.495493 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:06:36.495501 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:06:36.496701 (+ 1200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:06:36.497109 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:06:36.497110 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:06:36.497110 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:06:36.497168 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:06:36.497377 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:06:36.497377 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:06:36.497377 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:06:36.497484 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:06:36.497541 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0812 21:06:36.497792 (+ 251us) server_negotiation.cc:299] Negotiation successful
0812 21:06:36.497861 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260812 21:06:36.498168 6673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:36.489228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:38123 (local address 127.11.254.131:58323)
0812 21:06:36.489478 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0812 21:06:36.489510 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0812 21:06:36.490081 (+ 571us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:06:36.490310 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:06:36.490314 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:06:36.490971 (+ 657us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:06:36.491190 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:06:36.491195 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:06:36.491930 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:06:36.491932 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:06:36.492778 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:06:36.492781 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:06:36.493103 (+ 322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:06:36.493103 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:06:36.493228 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:06:36.493824 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:06:36.493834 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:06:36.495403 (+ 1569us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:06:36.496798 (+ 1395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:06:36.496805 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:06:36.496806 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:06:36.497043 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:06:36.497225 (+ 182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:06:36.497225 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:06:36.497226 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:06:36.497327 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:06:36.497544 (+ 217us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:06:36.497549 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:06:36.497679 (+ 130us) client_negotiation.cc:769] Sending connection context
0812 21:06:36.497766 (+ 87us) client_negotiation.cc:240] Negotiation successful
0812 21:06:36.497896 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":93,"threads_started":1}
I20260812 21:06:36.498698 6670 heartbeater.cc:347] Connected to a master server at 127.11.254.190:38123
I20260812 21:06:36.498801 6670 heartbeater.cc:464] Registering TS with master...
I20260812 21:06:36.498989 6670 heartbeater.cc:511] Master 127.11.254.190:38123 requested a full tablet report, sending...
I20260812 21:06:36.499382 5943 ts_manager.cc:194] Registered new tserver with Master: 024bc23e991c46bc813212a50b07f065 (127.11.254.131:38309)
I20260812 21:06:36.499768 5943 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.254.131@KRBTEST.COM'} at 127.11.254.131:58323
I20260812 21:06:36.505653 12282 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 21:06:36.513106 6040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:36.507075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55796 (local address 127.11.254.190:38123)
0812 21:06:36.507157 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0812 21:06:36.507159 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:06:36.507244 (+ 85us) server_negotiation.cc:407] Connection header received
0812 21:06:36.507312 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:06:36.507313 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:06:36.507324 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:06:36.507372 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:06:36.507927 (+ 555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:06:36.508445 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:06:36.509174 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:06:36.509377 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:06:36.509433 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:06:36.510611 (+ 1178us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:06:36.510612 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:06:36.510613 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:06:36.510620 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:06:36.511929 (+ 1309us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:06:36.512273 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:06:36.512274 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:06:36.512275 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:06:36.512342 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:06:36.512529 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:06:36.512530 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:06:36.512531 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:06:36.512703 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:06:36.512797 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0812 21:06:36.512927 (+ 130us) server_negotiation.cc:299] Negotiation successful
0812 21:06:36.513013 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 21:06:36.514906 5943 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55796:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 21:06:36.516661 5943 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 21:06:36.523509 6694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:36.519255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.131:38309 (local address 127.0.0.1:60372)
0812 21:06:36.519451 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0812 21:06:36.519495 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 21:06:36.519562 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:06:36.519989 (+ 427us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:06:36.519993 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:06:36.519996 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:06:36.520080 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:06:36.520083 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:06:36.521118 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:06:36.521119 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:06:36.522241 (+ 1122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:06:36.522245 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:06:36.523161 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:06:36.523162 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:06:36.523255 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:06:36.523293 (+ 38us) client_negotiation.cc:769] Sending connection context
0812 21:06:36.523363 (+ 70us) client_negotiation.cc:240] Negotiation successful
0812 21:06:36.523420 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":53,"threads_started":1}
I20260812 21:06:36.523523 6699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:36.519626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60372 (local address 127.11.254.131:38309)
0812 21:06:36.519805 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0812 21:06:36.519810 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 21:06:36.519821 (+ 11us) server_negotiation.cc:407] Connection header received
0812 21:06:36.519861 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:06:36.519863 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:06:36.519880 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:06:36.520158 (+ 278us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:06:36.520239 (+ 81us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:06:36.521012 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:06:36.522347 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:06:36.523053 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:06:36.523162 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:06:36.523224 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0812 21:06:36.523381 (+ 157us) server_negotiation.cc:299] Negotiation successful
0812 21:06:36.523424 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":90,"threads_started":1}
I20260812 21:06:36.523921 6697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:36.519300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.130:39975 (local address 127.0.0.1:47554)
0812 21:06:36.519814 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0812 21:06:36.519823 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 21:06:36.519880 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:06:36.520075 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:06:36.520134 (+ 59us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:06:36.520137 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:06:36.520199 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:06:36.520201 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:06:36.521669 (+ 1468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:06:36.521670 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:06:36.522686 (+ 1016us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:06:36.522689 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:06:36.523665 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:06:36.523666 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:06:36.523750 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:06:36.523755 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 21:06:36.523807 (+ 52us) client_negotiation.cc:240] Negotiation successful
0812 21:06:36.523848 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":494,"mutex_wait_us":27,"thread_start_us":93,"threads_started":1}
I20260812 21:06:36.523947 6696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:36.519461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47554 (local address 127.11.254.130:39975)
0812 21:06:36.519799 (+ 338us) server_negotiation.cc:206] Beginning negotiation
0812 21:06:36.519803 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 21:06:36.519869 (+ 66us) server_negotiation.cc:407] Connection header received
0812 21:06:36.519954 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:06:36.519957 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:06:36.519973 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:06:36.520277 (+ 304us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:06:36.520347 (+ 70us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:06:36.521472 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:06:36.522790 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:06:36.523513 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:06:36.523661 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:06:36.523726 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0812 21:06:36.523833 (+ 107us) server_negotiation.cc:299] Negotiation successful
0812 21:06:36.523872 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":306,"thread_start_us":83,"threads_started":1}
I20260812 21:06:36.524736 6695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:36.519460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.129:38685 (local address 127.0.0.1:37400)
0812 21:06:36.519978 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0812 21:06:36.519987 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 21:06:36.520028 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:06:36.520295 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:06:36.520296 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:06:36.520298 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:06:36.520343 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:06:36.520345 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:06:36.521687 (+ 1342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:06:36.521689 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:06:36.523549 (+ 1860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:06:36.523552 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:06:36.524508 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:06:36.524509 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:06:36.524570 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:06:36.524575 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 21:06:36.524628 (+ 53us) client_negotiation.cc:240] Negotiation successful
0812 21:06:36.524668 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":502,"mutex_wait_us":38,"thread_start_us":88,"threads_started":1}
I20260812 21:06:36.524780 6698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:36.519584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37400 (local address 127.11.254.129:38685)
0812 21:06:36.519769 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0812 21:06:36.519775 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 21:06:36.520086 (+ 311us) server_negotiation.cc:407] Connection header received
0812 21:06:36.520134 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:06:36.520137 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:06:36.520152 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:06:36.520214 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:06:36.520430 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:06:36.521527 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:06:36.523646 (+ 2119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:06:36.524342 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:06:36.524409 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:06:36.524472 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0812 21:06:36.524676 (+ 204us) server_negotiation.cc:299] Negotiation successful
0812 21:06:36.524717 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":95,"threads_started":1}
I20260812 21:06:36.524705 6573 tablet_service.cc:1467] Processing CreateTablet for tablet 834b8470a47c4ee6a83492d58fdc4f51 (DEFAULT_TABLE table=test-table [id=6bd6767f29e84eafb7d9ee9d904f5b0b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:06:36.525125 6573 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 834b8470a47c4ee6a83492d58fdc4f51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:06:36.526456 6349 tablet_service.cc:1467] Processing CreateTablet for tablet 834b8470a47c4ee6a83492d58fdc4f51 (DEFAULT_TABLE table=test-table [id=6bd6767f29e84eafb7d9ee9d904f5b0b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:06:36.526774 6349 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 834b8470a47c4ee6a83492d58fdc4f51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:06:36.528085 6700 tablet_bootstrap.cc:492] T 834b8470a47c4ee6a83492d58fdc4f51 P 024bc23e991c46bc813212a50b07f065: Bootstrap starting.
I20260812 21:06:36.528651 6700 tablet_bootstrap.cc:654] T 834b8470a47c4ee6a83492d58fdc4f51 P 024bc23e991c46bc813212a50b07f065: Neither blocks nor log segments found. Creating new log.
I20260812 21:06:36.529157 6700 log.cc:824] T 834b8470a47c4ee6a83492d58fdc4f51 P 024bc23e991c46bc813212a50b07f065: Log is configured to *not* fsync() on all Append() calls
I20260812 21:06:36.529526 6704 tablet_bootstrap.cc:492] T 834b8470a47c4ee6a83492d58fdc4f51 P 8eb6c52ef06a4d5f97762c59705d0047: Bootstrap starting.
I20260812 21:06:36.529994 6700 tablet_bootstrap.cc:492] T 834b8470a47c4ee6a83492d58fdc4f51 P 024bc23e991c46bc813212a50b07f065: No bootstrap required, opened a new log
I20260812 21:06:36.530063 6700 ts_tablet_manager.cc:1397] T 834b8470a47c4ee6a83492d58fdc4f51 P 024bc23e991c46bc813212a50b07f065: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 21:06:36.530385 6141 tablet_service.cc:1467] Processing CreateTablet for tablet 834b8470a47c4ee6a83492d58fdc4f51 (DEFAULT_TABLE table=test-table [id=6bd6767f29e84eafb7d9ee9d904f5b0b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:06:36.530786 6141 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 834b8470a47c4ee6a83492d58fdc4f51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:06:36.531575 6700 raft_consensus.cc:348] T 834b8470a47c4ee6a83492d58fdc4f51 P 024bc23e991c46bc813212a50b07f065 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "024bc23e991c46bc813212a50b07f065" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 38309 } } peers { permanent_uuid: "813f1c245a4e49c499d584012bf01a52" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 38685 } } peers { permanent_uuid: "8eb6c52ef06a4d5f97762c59705d0047" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 39975 } }
I20260812 21:06:36.531710 6700 raft_consensus.cc:374] T 834b8470a47c4ee6a83492d58fdc4f51 P 024bc23e991c46bc813212a50b07f065 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:06:36.531731 6700 raft_consensus.cc:729] T 834b8470a47c4ee6a83492d58fdc4f51 P 024bc23e991c46bc813212a50b07f065 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 024bc23e991c46bc813212a50b07f065, State: Initialized, Role: FOLLOWER
I20260812 21:06:36.531818 6700 consensus_queue.cc:260] T 834b8470a47c4ee6a83492d58fdc4f51 P 024bc23e991c46bc813212a50b07f065 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "024bc23e991c46bc813212a50b07f065" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 38309 } } peers { permanent_uuid: "813f1c245a4e49c499d584012bf01a52" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 38685 } } peers { permanent_uuid: "8eb6c52ef06a4d5f97762c59705d0047" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 39975 } }
I20260812 21:06:36.532065 6700 ts_tablet_manager.cc:1428] T 834b8470a47c4ee6a83492d58fdc4f51 P 024bc23e991c46bc813212a50b07f065: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260812 21:06:36.532147 6670 heartbeater.cc:503] Master 127.11.254.190:38123 was elected leader, sending a full tablet report...
I20260812 21:06:36.532661 6704 tablet_bootstrap.cc:654] T 834b8470a47c4ee6a83492d58fdc4f51 P 8eb6c52ef06a4d5f97762c59705d0047: Neither blocks nor log segments found. Creating new log.
I20260812 21:06:36.533087 6704 log.cc:824] T 834b8470a47c4ee6a83492d58fdc4f51 P 8eb6c52ef06a4d5f97762c59705d0047: Log is configured to *not* fsync() on all Append() calls
I20260812 21:06:36.533444 6707 tablet_bootstrap.cc:492] T 834b8470a47c4ee6a83492d58fdc4f51 P 813f1c245a4e49c499d584012bf01a52: Bootstrap starting.
I20260812 21:06:36.534045 6704 tablet_bootstrap.cc:492] T 834b8470a47c4ee6a83492d58fdc4f51 P 8eb6c52ef06a4d5f97762c59705d0047: No bootstrap required, opened a new log
I20260812 21:06:36.534113 6707 tablet_bootstrap.cc:654] T 834b8470a47c4ee6a83492d58fdc4f51 P 813f1c245a4e49c499d584012bf01a52: Neither blocks nor log segments found. Creating new log.
I20260812 21:06:36.534118 6704 ts_tablet_manager.cc:1397] T 834b8470a47c4ee6a83492d58fdc4f51 P 8eb6c52ef06a4d5f97762c59705d0047: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260812 21:06:36.534662 6707 log.cc:824] T 834b8470a47c4ee6a83492d58fdc4f51 P 813f1c245a4e49c499d584012bf01a52: Log is configured to *not* fsync() on all Append() calls
I20260812 21:06:36.535422 6707 tablet_bootstrap.cc:492] T 834b8470a47c4ee6a83492d58fdc4f51 P 813f1c245a4e49c499d584012bf01a52: No bootstrap required, opened a new log
I20260812 21:06:36.535496 6707 ts_tablet_manager.cc:1397] T 834b8470a47c4ee6a83492d58fdc4f51 P 813f1c245a4e49c499d584012bf01a52: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260812 21:06:36.536170 6704 raft_consensus.cc:348] T 834b8470a47c4ee6a83492d58fdc4f51 P 8eb6c52ef06a4d5f97762c59705d0047 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "024bc23e991c46bc813212a50b07f065" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 38309 } } peers { permanent_uuid: "813f1c245a4e49c499d584012bf01a52" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 38685 } } peers { permanent_uuid: "8eb6c52ef06a4d5f97762c59705d0047" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 39975 } }
I20260812 21:06:36.536365 6704 raft_consensus.cc:374] T 834b8470a47c4ee6a83492d58fdc4f51 P 8eb6c52ef06a4d5f97762c59705d0047 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:06:36.536412 6704 raft_consensus.cc:729] T 834b8470a47c4ee6a83492d58fdc4f51 P 8eb6c52ef06a4d5f97762c59705d0047 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8eb6c52ef06a4d5f97762c59705d0047, State: Initialized, Role: FOLLOWER
I20260812 21:06:36.536536 6704 consensus_queue.cc:260] T 834b8470a47c4ee6a83492d58fdc4f51 P 8eb6c52ef06a4d5f97762c59705d0047 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "024bc23e991c46bc813212a50b07f065" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 38309 } } peers { permanent_uuid: "813f1c245a4e49c499d584012bf01a52" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 38685 } } peers { permanent_uuid: "8eb6c52ef06a4d5f97762c59705d0047" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 39975 } }
I20260812 21:06:36.536831 6704 ts_tablet_manager.cc:1428] T 834b8470a47c4ee6a83492d58fdc4f51 P 8eb6c52ef06a4d5f97762c59705d0047: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 21:06:36.536852 6460 heartbeater.cc:503] Master 127.11.254.190:38123 was elected leader, sending a full tablet report...
I20260812 21:06:36.537348 6707 raft_consensus.cc:348] T 834b8470a47c4ee6a83492d58fdc4f51 P 813f1c245a4e49c499d584012bf01a52 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "024bc23e991c46bc813212a50b07f065" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 38309 } } peers { permanent_uuid: "813f1c245a4e49c499d584012bf01a52" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 38685 } } peers { permanent_uuid: "8eb6c52ef06a4d5f97762c59705d0047" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 39975 } }
I20260812 21:06:36.537514 6707 raft_consensus.cc:374] T 834b8470a47c4ee6a83492d58fdc4f51 P 813f1c245a4e49c499d584012bf01a52 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:06:36.537554 6707 raft_consensus.cc:729] T 834b8470a47c4ee6a83492d58fdc4f51 P 813f1c245a4e49c499d584012bf01a52 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 813f1c245a4e49c499d584012bf01a52, State: Initialized, Role: FOLLOWER
I20260812 21:06:36.537678 6707 consensus_queue.cc:260] T 834b8470a47c4ee6a83492d58fdc4f51 P 813f1c245a4e49c499d584012bf01a52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "024bc23e991c46bc813212a50b07f065" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 38309 } } peers { permanent_uuid: "813f1c245a4e49c499d584012bf01a52" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 38685 } } peers { permanent_uuid: "8eb6c52ef06a4d5f97762c59705d0047" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 39975 } }
I20260812 21:06:36.537936 6707 ts_tablet_manager.cc:1428] T 834b8470a47c4ee6a83492d58fdc4f51 P 813f1c245a4e49c499d584012bf01a52: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260812 21:06:36.537971 6241 heartbeater.cc:503] Master 127.11.254.190:38123 was elected leader, sending a full tablet report...
I20260812 21:06:36.575038 6711 raft_consensus.cc:482] T 834b8470a47c4ee6a83492d58fdc4f51 P 8eb6c52ef06a4d5f97762c59705d0047 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 21:06:36.575198 6711 raft_consensus.cc:504] T 834b8470a47c4ee6a83492d58fdc4f51 P 8eb6c52ef06a4d5f97762c59705d0047 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "024bc23e991c46bc813212a50b07f065" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 38309 } } peers { permanent_uuid: "813f1c245a4e49c499d584012bf01a52" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 38685 } } peers { permanent_uuid: "8eb6c52ef06a4d5f97762c59705d0047" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 39975 } }
W20260812 21:06:36.575385 6464 tablet.cc:2369] T 834b8470a47c4ee6a83492d58fdc4f51 P 8eb6c52ef06a4d5f97762c59705d0047: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 21:06:36.575661 6711 leader_election.cc:288] T 834b8470a47c4ee6a83492d58fdc4f51 P 8eb6c52ef06a4d5f97762c59705d0047 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 024bc23e991c46bc813212a50b07f065 (127.11.254.131:38309), 813f1c245a4e49c499d584012bf01a52 (127.11.254.129:38685)
I20260812 21:06:36.578809 6699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:36.575983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.130:33081 (local address 127.11.254.131:38309)
0812 21:06:36.576072 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0812 21:06:36.576075 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 21:06:36.576084 (+ 9us) server_negotiation.cc:407] Connection header received
0812 21:06:36.576113 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:06:36.576115 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:06:36.576124 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:06:36.576175 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:06:36.576356 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:06:36.577055 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:06:36.578048 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:06:36.578453 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:06:36.578523 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:06:36.578534 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0812 21:06:36.578702 (+ 168us) server_negotiation.cc:299] Negotiation successful
0812 21:06:36.578745 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260812 21:06:36.578809 6465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:36.575873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.131:38309 (local address 127.11.254.130:33081)
0812 21:06:36.575935 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0812 21:06:36.575948 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 21:06:36.576028 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:06:36.576201 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:06:36.576202 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:06:36.576204 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:06:36.576285 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:06:36.576287 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:06:36.577141 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:06:36.577143 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:06:36.577940 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:06:36.577943 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:06:36.578543 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:06:36.578545 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:06:36.578628 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:06:36.578632 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 21:06:36.578678 (+ 46us) client_negotiation.cc:240] Negotiation successful
0812 21:06:36.578725 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35,"mutex_wait_us":26}
I20260812 21:06:36.578933 6716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:36.575995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.129:38685 (local address 127.11.254.130:57981)
0812 21:06:36.576201 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0812 21:06:36.576212 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 21:06:36.576279 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:06:36.576459 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:06:36.576460 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:06:36.576461 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:06:36.576528 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:06:36.576529 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:06:36.577268 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:06:36.577269 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:06:36.578059 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:06:36.578061 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:06:36.578646 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:06:36.578648 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:06:36.578724 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:06:36.578727 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 21:06:36.578773 (+ 46us) client_negotiation.cc:240] Negotiation successful
0812 21:06:36.578816 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":140,"threads_started":1}
I20260812 21:06:36.579070 6698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:36.576116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.130:57981 (local address 127.11.254.129:38685)
0812 21:06:36.576166 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0812 21:06:36.576168 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:06:36.576280 (+ 112us) server_negotiation.cc:407] Connection header received
0812 21:06:36.576344 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:06:36.576345 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:06:36.576356 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:06:36.576411 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:06:36.576593 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:06:36.577185 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:06:36.578160 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:06:36.578556 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:06:36.578622 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:06:36.578630 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 21:06:36.578798 (+ 168us) server_negotiation.cc:299] Negotiation successful
0812 21:06:36.578840 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260812 21:06:36.579280 6599 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "834b8470a47c4ee6a83492d58fdc4f51" candidate_uuid: "8eb6c52ef06a4d5f97762c59705d0047" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "024bc23e991c46bc813212a50b07f065" is_pre_election: true
I20260812 21:06:36.579345 6177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "834b8470a47c4ee6a83492d58fdc4f51" candidate_uuid: "8eb6c52ef06a4d5f97762c59705d0047" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "813f1c245a4e49c499d584012bf01a52" is_pre_election: true
I20260812 21:06:36.579448 6599 raft_consensus.cc:2454] T 834b8470a47c4ee6a83492d58fdc4f51 P 024bc23e991c46bc813212a50b07f065 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8eb6c52ef06a4d5f97762c59705d0047 in term 0.
I20260812 21:06:36.579504 6177 raft_consensus.cc:2454] T 834b8470a47c4ee6a83492d58fdc4f51 P 813f1c245a4e49c499d584012bf01a52 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8eb6c52ef06a4d5f97762c59705d0047 in term 0.
I20260812 21:06:36.579720 6269 leader_election.cc:302] T 834b8470a47c4ee6a83492d58fdc4f51 P 8eb6c52ef06a4d5f97762c59705d0047 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 024bc23e991c46bc813212a50b07f065, 8eb6c52ef06a4d5f97762c59705d0047; no voters:
I20260812 21:06:36.579859 6711 raft_consensus.cc:2781] T 834b8470a47c4ee6a83492d58fdc4f51 P 8eb6c52ef06a4d5f97762c59705d0047 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 21:06:36.579928 6711 raft_consensus.cc:482] T 834b8470a47c4ee6a83492d58fdc4f51 P 8eb6c52ef06a4d5f97762c59705d0047 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 21:06:36.579952 6711 raft_consensus.cc:3037] T 834b8470a47c4ee6a83492d58fdc4f51 P 8eb6c52ef06a4d5f97762c59705d0047 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:06:36.580713 6711 raft_consensus.cc:504] T 834b8470a47c4ee6a83492d58fdc4f51 P 8eb6c52ef06a4d5f97762c59705d0047 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "024bc23e991c46bc813212a50b07f065" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 38309 } } peers { permanent_uuid: "813f1c245a4e49c499d584012bf01a52" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 38685 } } peers { permanent_uuid: "8eb6c52ef06a4d5f97762c59705d0047" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 39975 } }
I20260812 21:06:36.580850 6711 leader_election.cc:288] T 834b8470a47c4ee6a83492d58fdc4f51 P 8eb6c52ef06a4d5f97762c59705d0047 [CANDIDATE]: Term 1 election: Requested vote from peers 024bc23e991c46bc813212a50b07f065 (127.11.254.131:38309), 813f1c245a4e49c499d584012bf01a52 (127.11.254.129:38685)
I20260812 21:06:36.581064 6177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "834b8470a47c4ee6a83492d58fdc4f51" candidate_uuid: "8eb6c52ef06a4d5f97762c59705d0047" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "813f1c245a4e49c499d584012bf01a52"
I20260812 21:06:36.581049 6599 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "834b8470a47c4ee6a83492d58fdc4f51" candidate_uuid: "8eb6c52ef06a4d5f97762c59705d0047" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "024bc23e991c46bc813212a50b07f065"
I20260812 21:06:36.581156 6599 raft_consensus.cc:3037] T 834b8470a47c4ee6a83492d58fdc4f51 P 024bc23e991c46bc813212a50b07f065 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:06:36.581156 6177 raft_consensus.cc:3037] T 834b8470a47c4ee6a83492d58fdc4f51 P 813f1c245a4e49c499d584012bf01a52 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:06:36.582149 6599 raft_consensus.cc:2454] T 834b8470a47c4ee6a83492d58fdc4f51 P 024bc23e991c46bc813212a50b07f065 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8eb6c52ef06a4d5f97762c59705d0047 in term 1.
I20260812 21:06:36.582149 6177 raft_consensus.cc:2454] T 834b8470a47c4ee6a83492d58fdc4f51 P 813f1c245a4e49c499d584012bf01a52 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8eb6c52ef06a4d5f97762c59705d0047 in term 1.
I20260812 21:06:36.582356 6269 leader_election.cc:302] T 834b8470a47c4ee6a83492d58fdc4f51 P 8eb6c52ef06a4d5f97762c59705d0047 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 813f1c245a4e49c499d584012bf01a52, 8eb6c52ef06a4d5f97762c59705d0047; no voters:
I20260812 21:06:36.582442 6711 raft_consensus.cc:2781] T 834b8470a47c4ee6a83492d58fdc4f51 P 8eb6c52ef06a4d5f97762c59705d0047 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:06:36.582633 6711 raft_consensus.cc:686] T 834b8470a47c4ee6a83492d58fdc4f51 P 8eb6c52ef06a4d5f97762c59705d0047 [term 1 LEADER]: Becoming Leader. State: Replica: 8eb6c52ef06a4d5f97762c59705d0047, State: Running, Role: LEADER
I20260812 21:06:36.582731 6711 consensus_queue.cc:237] T 834b8470a47c4ee6a83492d58fdc4f51 P 8eb6c52ef06a4d5f97762c59705d0047 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "024bc23e991c46bc813212a50b07f065" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 38309 } } peers { permanent_uuid: "813f1c245a4e49c499d584012bf01a52" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 38685 } } peers { permanent_uuid: "8eb6c52ef06a4d5f97762c59705d0047" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 39975 } }
I20260812 21:06:36.583779 5943 catalog_manager.cc:5495] T 834b8470a47c4ee6a83492d58fdc4f51 P 8eb6c52ef06a4d5f97762c59705d0047 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8eb6c52ef06a4d5f97762c59705d0047 (127.11.254.130). New cstate: current_term: 1 leader_uuid: "8eb6c52ef06a4d5f97762c59705d0047" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "024bc23e991c46bc813212a50b07f065" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 38309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "813f1c245a4e49c499d584012bf01a52" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 38685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8eb6c52ef06a4d5f97762c59705d0047" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 39975 } health_report { overall_health: HEALTHY } } }
I20260812 21:06:36.596452 6696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:36.594020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47570 (local address 127.11.254.130:39975)
0812 21:06:36.594078 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0812 21:06:36.594080 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:06:36.594089 (+ 9us) server_negotiation.cc:407] Connection header received
0812 21:06:36.594141 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:06:36.594142 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:06:36.594163 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:06:36.594225 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 21:06:36.594396 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:06:36.594907 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:06:36.595663 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:06:36.595858 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:06:36.595910 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:06:36.596045 (+ 135us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 21:06:36.596151 (+ 106us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 21:06:36.596210 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0812 21:06:36.596322 (+ 112us) server_negotiation.cc:299] Negotiation successful
0812 21:06:36.596342 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260812 21:06:36.598335 6599 raft_consensus.cc:1261] T 834b8470a47c4ee6a83492d58fdc4f51 P 024bc23e991c46bc813212a50b07f065 [term 1 FOLLOWER]: Refusing update from remote peer 8eb6c52ef06a4d5f97762c59705d0047: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 21:06:36.598330 6177 raft_consensus.cc:1261] T 834b8470a47c4ee6a83492d58fdc4f51 P 813f1c245a4e49c499d584012bf01a52 [term 1 FOLLOWER]: Refusing update from remote peer 8eb6c52ef06a4d5f97762c59705d0047: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 21:06:36.598728 6717 consensus_queue.cc:1035] T 834b8470a47c4ee6a83492d58fdc4f51 P 8eb6c52ef06a4d5f97762c59705d0047 [LEADER]: Connected to new peer: Peer: permanent_uuid: "813f1c245a4e49c499d584012bf01a52" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 38685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 21:06:36.598898 6711 consensus_queue.cc:1035] T 834b8470a47c4ee6a83492d58fdc4f51 P 8eb6c52ef06a4d5f97762c59705d0047 [LEADER]: Connected to new peer: Peer: permanent_uuid: "024bc23e991c46bc813212a50b07f065" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 38309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 21:06:36.601656 6725 mvcc.cc:205] Tried to move back new op lower bound from 7317785790863876096 to 7317785790803496960. Current Snapshot: MvccSnapshot[applied={T|T < 7317785790863876096}]
I20260812 21:06:36.602331 6724 mvcc.cc:205] Tried to move back new op lower bound from 7317785790863876096 to 7317785790803496960. Current Snapshot: MvccSnapshot[applied={T|T < 7317785790863876096}]
I20260812 21:06:36.603969 5942 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55796:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 21:06:36.604067 5942 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55796:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 21:06:36.604892 5942 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 36a27cabe4ae44568b7adc99d86a1430: Sending DeleteTablet for 3 replicas of tablet 834b8470a47c4ee6a83492d58fdc4f51
I20260812 21:06:36.605250 6349 tablet_service.cc:1514] Processing DeleteTablet for tablet 834b8470a47c4ee6a83492d58fdc4f51 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:06:36 UTC) from {username='kudu', principal='kudu/127.11.254.190@KRBTEST.COM'} at 127.0.0.1:47554
I20260812 21:06:36.605297 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 6047
I20260812 21:06:36.605346 6573 tablet_service.cc:1514] Processing DeleteTablet for tablet 834b8470a47c4ee6a83492d58fdc4f51 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:06:36 UTC) from {username='kudu', principal='kudu/127.11.254.190@KRBTEST.COM'} at 127.0.0.1:60372
I20260812 21:06:36.618014 6735 tablet_replica.cc:330] T 834b8470a47c4ee6a83492d58fdc4f51 P 8eb6c52ef06a4d5f97762c59705d0047: stopping tablet replica
I20260812 21:06:36.620640 6735 raft_consensus.cc:2229] T 834b8470a47c4ee6a83492d58fdc4f51 P 8eb6c52ef06a4d5f97762c59705d0047 [term 1 LEADER]: Raft consensus shutting down.
I20260812 21:06:36.620930 6735 raft_consensus.cc:2258] T 834b8470a47c4ee6a83492d58fdc4f51 P 8eb6c52ef06a4d5f97762c59705d0047 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 21:06:36.626647 6735 ts_tablet_manager.cc:1905] T 834b8470a47c4ee6a83492d58fdc4f51 P 8eb6c52ef06a4d5f97762c59705d0047: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 21:06:36.628633 6736 tablet_replica.cc:330] T 834b8470a47c4ee6a83492d58fdc4f51 P 024bc23e991c46bc813212a50b07f065: stopping tablet replica
I20260812 21:06:36.628872 6736 raft_consensus.cc:2229] T 834b8470a47c4ee6a83492d58fdc4f51 P 024bc23e991c46bc813212a50b07f065 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 21:06:36.629115 6736 raft_consensus.cc:2258] T 834b8470a47c4ee6a83492d58fdc4f51 P 024bc23e991c46bc813212a50b07f065 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 21:06:36.631450 6735 ts_tablet_manager.cc:1918] T 834b8470a47c4ee6a83492d58fdc4f51 P 8eb6c52ef06a4d5f97762c59705d0047: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 21:06:36.631712 6735 log.cc:1196] T 834b8470a47c4ee6a83492d58fdc4f51 P 8eb6c52ef06a4d5f97762c59705d0047: Deleting WAL directory at /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/ts-1/wal/wals/834b8470a47c4ee6a83492d58fdc4f51
I20260812 21:06:36.632100 6735 ts_tablet_manager.cc:1939] T 834b8470a47c4ee6a83492d58fdc4f51 P 8eb6c52ef06a4d5f97762c59705d0047: Deleting consensus metadata
I20260812 21:06:36.632695 5927 catalog_manager.cc:4841] TS 8eb6c52ef06a4d5f97762c59705d0047 (127.11.254.130:39975): tablet 834b8470a47c4ee6a83492d58fdc4f51 (table test-table [id=6bd6767f29e84eafb7d9ee9d904f5b0b]) successfully deleted
I20260812 21:06:36.633622 6736 ts_tablet_manager.cc:1905] T 834b8470a47c4ee6a83492d58fdc4f51 P 024bc23e991c46bc813212a50b07f065: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 21:06:36.635519 6736 ts_tablet_manager.cc:1918] T 834b8470a47c4ee6a83492d58fdc4f51 P 024bc23e991c46bc813212a50b07f065: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 21:06:36.635687 6736 log.cc:1196] T 834b8470a47c4ee6a83492d58fdc4f51 P 024bc23e991c46bc813212a50b07f065: Deleting WAL directory at /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/ts-2/wal/wals/834b8470a47c4ee6a83492d58fdc4f51
I20260812 21:06:36.636073 6736 ts_tablet_manager.cc:1939] T 834b8470a47c4ee6a83492d58fdc4f51 P 024bc23e991c46bc813212a50b07f065: Deleting consensus metadata
I20260812 21:06:36.638775 5923 catalog_manager.cc:4841] TS 024bc23e991c46bc813212a50b07f065 (127.11.254.131:38309): tablet 834b8470a47c4ee6a83492d58fdc4f51 (table test-table [id=6bd6767f29e84eafb7d9ee9d904f5b0b]) successfully deleted
W20260812 21:06:36.643916 5923 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.254.129:38685 (error 108)
I20260812 21:06:36.644162 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 6247
I20260812 21:06:36.644551 6694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:36.644418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.129:38685 (local address 127.0.0.1:37402)
0812 21:06:36.644484 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0812 21:06:36.644513 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.129:38685: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260812 21:06:36.647140 5923 catalog_manager.cc:4584] TS 813f1c245a4e49c499d584012bf01a52 (127.11.254.129:38685): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 834b8470a47c4ee6a83492d58fdc4f51: Network error: Client connection negotiation failed: client connection to 127.11.254.129:38685: connect: Connection refused (error 111)
I20260812 21:06:36.661444 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 6468
I20260812 21:06:36.676915 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 5897
I20260812 21:06:56.689088 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.254.190:38123
--webserver_interface=127.11.254.190
--webserver_port=33283
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.190.keytab
--principal=kudu/127.11.254.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.254.148:34711
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.254.190:38123
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:06:56.773643 6740 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:06:56.773854 6740 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:06:56.773874 6740 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:06:56.775024 6740 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260812 21:06:56.775064 6740 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260812 21:06:56.775077 6740 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260812 21:06:56.775087 6740 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 21:06:56.775098 6740 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:06:56.775110 6740 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:06:56.775125 6740 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 21:06:56.775137 6740 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 21:06:56.776647 6740 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.254.148:34711
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.11.254.190:38123
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.190.keytab
--principal=kudu/127.11.254.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.254.190:38123
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.254.190
--webserver_port=33283
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6740
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:06:56.776847 6740 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:06:56.777048 6740 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:06:56.786688 6747 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:06:56.786725 6748 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:06:56.786726 6751 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:06:56.786746 6740 server_base.cc:1034] running on GCE node
I20260812 21:06:56.787268 6740 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:06:56.787602 6740 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:06:56.788771 6740 hybrid_clock.cc:648] HybridClock initialized: now 1786568816788754 us; error 30 us; skew 500 ppm
Aug 12 21:06:56 dist-test-slave-vpvm krb5kdc[5864](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568816, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:06:56.791641 6740 init.cc:377] Logged in from keytab as kudu/127.11.254.190@KRBTEST.COM (short username kudu)
I20260812 21:06:56.792196 6740 webserver.cc:460] Webserver started at http://127.11.254.190:33283/ using document root <none> and password file <none>
I20260812 21:06:56.792505 6740 fs_manager.cc:359] Metadata directory not provided
I20260812 21:06:56.792582 6740 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:06:56.794085 6740 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 21:06:56.794806 6765 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:06:56.794965 6740 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 21:06:56.795073 6740 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/master-0/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/master-0/wal
uuid: "36a27cabe4ae44568b7adc99d86a1430"
format_stamp: "Formatted at 2026-08-12 21:06:35 on dist-test-slave-vpvm"
server_key: "0c538b3805728f70b6f07f44ee6460b6"
server_key_iv: "5f1b651e34f771fe03d7352717eea675"
server_key_version: "encryptionkey@0"
I20260812 21:06:56.795516 6740 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:06:56.820147 6740 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:06:56.822463 6740 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:06:56.822595 6740 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:06:56.826807 6740 rpc_server.cc:307] RPC server started. Bound to: 127.11.254.190:38123
I20260812 21:06:56.826840 6848 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.254.190:38123 every 8 connection(s)
I20260812 21:06:56.827224 6740 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
I20260812 21:06:56.828840 6849 sys_catalog.cc:263] Verifying existing consensus state
I20260812 21:06:56.829559 6849 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36a27cabe4ae44568b7adc99d86a1430: Bootstrap starting.
I20260812 21:06:56.836292 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 6740
I20260812 21:06:56.836666 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.254.129:38685
--local_ip_for_outbound_sockets=127.11.254.129
--tserver_master_addrs=127.11.254.190:38123
--webserver_port=34443
--webserver_interface=127.11.254.129
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.129.keytab
--principal=kudu/127.11.254.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.254.148:34711
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 21:06:56.839715 6849 log.cc:824] T 00000000000000000000000000000000 P 36a27cabe4ae44568b7adc99d86a1430: Log is configured to *not* fsync() on all Append() calls
I20260812 21:06:56.841982 6849 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36a27cabe4ae44568b7adc99d86a1430: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 21:06:56.842285 6849 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36a27cabe4ae44568b7adc99d86a1430: Bootstrap complete.
I20260812 21:06:56.844568 6849 raft_consensus.cc:348] T 00000000000000000000000000000000 P 36a27cabe4ae44568b7adc99d86a1430 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36a27cabe4ae44568b7adc99d86a1430" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 38123 } }
I20260812 21:06:56.844811 6849 raft_consensus.cc:729] T 00000000000000000000000000000000 P 36a27cabe4ae44568b7adc99d86a1430 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36a27cabe4ae44568b7adc99d86a1430, State: Initialized, Role: FOLLOWER
I20260812 21:06:56.844981 6849 consensus_queue.cc:260] T 00000000000000000000000000000000 P 36a27cabe4ae44568b7adc99d86a1430 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36a27cabe4ae44568b7adc99d86a1430" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 38123 } }
I20260812 21:06:56.845062 6849 raft_consensus.cc:388] T 00000000000000000000000000000000 P 36a27cabe4ae44568b7adc99d86a1430 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 21:06:56.845119 6849 raft_consensus.cc:482] T 00000000000000000000000000000000 P 36a27cabe4ae44568b7adc99d86a1430 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 21:06:56.845170 6849 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 36a27cabe4ae44568b7adc99d86a1430 [term 1 FOLLOWER]: Advancing to term 2
I20260812 21:06:56.846289 6849 raft_consensus.cc:504] T 00000000000000000000000000000000 P 36a27cabe4ae44568b7adc99d86a1430 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36a27cabe4ae44568b7adc99d86a1430" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 38123 } }
I20260812 21:06:56.846391 6849 leader_election.cc:302] T 00000000000000000000000000000000 P 36a27cabe4ae44568b7adc99d86a1430 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 36a27cabe4ae44568b7adc99d86a1430; no voters:
I20260812 21:06:56.846577 6849 leader_election.cc:288] T 00000000000000000000000000000000 P 36a27cabe4ae44568b7adc99d86a1430 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260812 21:06:56.846627 6854 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 36a27cabe4ae44568b7adc99d86a1430 [term 2 FOLLOWER]: Leader election won for term 2
I20260812 21:06:56.846786 6854 raft_consensus.cc:686] T 00000000000000000000000000000000 P 36a27cabe4ae44568b7adc99d86a1430 [term 2 LEADER]: Becoming Leader. State: Replica: 36a27cabe4ae44568b7adc99d86a1430, State: Running, Role: LEADER
I20260812 21:06:56.846863 6849 sys_catalog.cc:564] T 00000000000000000000000000000000 P 36a27cabe4ae44568b7adc99d86a1430 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 21:06:56.846906 6854 consensus_queue.cc:237] T 00000000000000000000000000000000 P 36a27cabe4ae44568b7adc99d86a1430 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36a27cabe4ae44568b7adc99d86a1430" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 38123 } }
I20260812 21:06:56.847201 6854 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36a27cabe4ae44568b7adc99d86a1430 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 36a27cabe4ae44568b7adc99d86a1430. Latest consensus state: current_term: 2 leader_uuid: "36a27cabe4ae44568b7adc99d86a1430" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36a27cabe4ae44568b7adc99d86a1430" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 38123 } } }
I20260812 21:06:56.847282 6856 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36a27cabe4ae44568b7adc99d86a1430 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "36a27cabe4ae44568b7adc99d86a1430" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36a27cabe4ae44568b7adc99d86a1430" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 38123 } } }
I20260812 21:06:56.847363 6854 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36a27cabe4ae44568b7adc99d86a1430 [sys.catalog]: This master's current role is: LEADER
I20260812 21:06:56.847805 6856 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36a27cabe4ae44568b7adc99d86a1430 [sys.catalog]: This master's current role is: LEADER
I20260812 21:06:56.848008 6868 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 21:06:56.848831 6868 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 21:06:56.849023 6868 catalog_manager.cc:1259] Loaded cluster ID: 27a149e698fb48b0a6d33906f55215fa
I20260812 21:06:56.849197 6868 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 21:06:56.850355 6868 catalog_manager.cc:1504] Loading token signing keys...
I20260812 21:06:56.850561 6868 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 36a27cabe4ae44568b7adc99d86a1430: Loaded TSK: 0
I20260812 21:06:56.872272 6868 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 36a27cabe4ae44568b7adc99d86a1430: Generated new TSK 1
I20260812 21:06:56.872399 6868 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 21:06:56.927263 6852 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:06:56.927536 6852 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:06:56.927582 6852 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:06:56.928785 6852 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:06:56.928831 6852 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:06:56.928866 6852 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.254.129
I20260812 21:06:56.930429 6852 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.254.148:34711
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.129.keytab
--principal=kudu/127.11.254.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.254.129:38685
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.254.129
--webserver_port=34443
--tserver_master_addrs=127.11.254.190:38123
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6852
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.254.129
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:06:56.930744 6852 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:06:56.930976 6852 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:06:56.936203 6897 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:06:56.936239 6891 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:06:56.936228 6893 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:06:56.936785 6852 server_base.cc:1034] running on GCE node
I20260812 21:06:56.937017 6852 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:06:56.937256 6852 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:06:56.938477 6852 hybrid_clock.cc:648] HybridClock initialized: now 1786568816938462 us; error 33 us; skew 500 ppm
Aug 12 21:06:56 dist-test-slave-vpvm krb5kdc[5864](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568816, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:06:56.941211 6852 init.cc:377] Logged in from keytab as kudu/127.11.254.129@KRBTEST.COM (short username kudu)
I20260812 21:06:56.941751 6852 webserver.cc:460] Webserver started at http://127.11.254.129:34443/ using document root <none> and password file <none>
I20260812 21:06:56.942032 6852 fs_manager.cc:359] Metadata directory not provided
I20260812 21:06:56.942119 6852 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:06:56.943660 6852 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 21:06:56.944388 6911 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:06:56.944588 6852 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 21:06:56.944697 6852 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/ts-0/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/ts-0/wal
uuid: "813f1c245a4e49c499d584012bf01a52"
format_stamp: "Formatted at 2026-08-12 21:06:36 on dist-test-slave-vpvm"
server_key: "a606426d65f79390689a46a5466b23a5"
server_key_iv: "efa852fc71081e51e3bd8973ee227f20"
server_key_version: "encryptionkey@0"
I20260812 21:06:56.945190 6852 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:06:56.960157 6852 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:06:56.962929 6852 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:06:56.963111 6852 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:06:56.963434 6852 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:06:56.964017 6919 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260812 21:06:56.965629 6852 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 21:06:56.965694 6852 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260812 21:06:56.965729 6852 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260812 21:06:56.966722 6852 ts_tablet_manager.cc:610] Registered 1 tablets
I20260812 21:06:56.966773 6852 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260812 21:06:56.966876 6919 tablet_bootstrap.cc:492] T 834b8470a47c4ee6a83492d58fdc4f51 P 813f1c245a4e49c499d584012bf01a52: Bootstrap starting.
I20260812 21:06:56.974485 6852 rpc_server.cc:307] RPC server started. Bound to: 127.11.254.129:38685
I20260812 21:06:56.974756 7086 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.254.129:38685 every 8 connection(s)
I20260812 21:06:56.975060 6852 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/ts-0/data/info.pb
I20260812 21:06:56.976240 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 6852
I20260812 21:06:56.976562 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.254.130:39975
--local_ip_for_outbound_sockets=127.11.254.130
--tserver_master_addrs=127.11.254.190:38123
--webserver_port=43347
--webserver_interface=127.11.254.130
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.130.keytab
--principal=kudu/127.11.254.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.254.148:34711
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 21:06:56.980877 6919 log.cc:824] T 834b8470a47c4ee6a83492d58fdc4f51 P 813f1c245a4e49c499d584012bf01a52: Log is configured to *not* fsync() on all Append() calls
I20260812 21:06:56.983847 6919 tablet_bootstrap.cc:492] T 834b8470a47c4ee6a83492d58fdc4f51 P 813f1c245a4e49c499d584012bf01a52: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 21:06:56.984179 6919 tablet_bootstrap.cc:492] T 834b8470a47c4ee6a83492d58fdc4f51 P 813f1c245a4e49c499d584012bf01a52: Bootstrap complete.
I20260812 21:06:56.985446 6919 ts_tablet_manager.cc:1397] T 834b8470a47c4ee6a83492d58fdc4f51 P 813f1c245a4e49c499d584012bf01a52: Time spent bootstrapping tablet: real 0.019s user 0.000s sys 0.014s
I20260812 21:06:56.987697 6919 raft_consensus.cc:348] T 834b8470a47c4ee6a83492d58fdc4f51 P 813f1c245a4e49c499d584012bf01a52 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "024bc23e991c46bc813212a50b07f065" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 38309 } } peers { permanent_uuid: "813f1c245a4e49c499d584012bf01a52" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 38685 } } peers { permanent_uuid: "8eb6c52ef06a4d5f97762c59705d0047" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 39975 } }
I20260812 21:06:56.988178 6919 raft_consensus.cc:729] T 834b8470a47c4ee6a83492d58fdc4f51 P 813f1c245a4e49c499d584012bf01a52 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 813f1c245a4e49c499d584012bf01a52, State: Initialized, Role: FOLLOWER
I20260812 21:06:56.988418 6919 consensus_queue.cc:260] T 834b8470a47c4ee6a83492d58fdc4f51 P 813f1c245a4e49c499d584012bf01a52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "024bc23e991c46bc813212a50b07f065" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 38309 } } peers { permanent_uuid: "813f1c245a4e49c499d584012bf01a52" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 38685 } } peers { permanent_uuid: "8eb6c52ef06a4d5f97762c59705d0047" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 39975 } }
Aug 12 21:06:56 dist-test-slave-vpvm krb5kdc[5864](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568816, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.129@KRBTEST.COM for kudu/127.11.254.190@KRBTEST.COM
I20260812 21:06:56.988848 6919 ts_tablet_manager.cc:1428] T 834b8470a47c4ee6a83492d58fdc4f51 P 813f1c245a4e49c499d584012bf01a52: Time spent starting tablet: real 0.003s user 0.003s sys 0.001s
I20260812 21:06:56.993587 7092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:56.975805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:38123 (local address 127.11.254.129:55161)
0812 21:06:56.975991 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0812 21:06:56.976037 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 21:06:56.976678 (+ 641us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:06:56.981845 (+ 5167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:06:56.981851 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:06:56.982259 (+ 408us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:06:56.982553 (+ 294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:06:56.982561 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:06:56.984344 (+ 1783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:06:56.984345 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:06:56.985646 (+ 1301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:06:56.985650 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:06:56.986559 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:06:56.986560 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:06:56.986707 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:06:56.987350 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:06:56.987360 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:06:56.989182 (+ 1822us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:06:56.991580 (+ 2398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:06:56.991589 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:06:56.991591 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:06:56.991918 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:06:56.992212 (+ 294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:06:56.992213 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:06:56.992213 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:06:56.992365 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:06:56.992867 (+ 502us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:06:56.992875 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:06:56.993045 (+ 170us) client_negotiation.cc:769] Sending connection context
0812 21:06:56.993164 (+ 119us) client_negotiation.cc:240] Negotiation successful
0812 21:06:56.993356 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":91,"threads_started":1}
I20260812 21:06:56.993593 7094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:56.975916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.129:55161 (local address 127.11.254.190:38123)
0812 21:06:56.981321 (+ 5405us) server_negotiation.cc:206] Beginning negotiation
0812 21:06:56.981331 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 21:06:56.981384 (+ 53us) server_negotiation.cc:407] Connection header received
0812 21:06:56.981549 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:06:56.981556 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:06:56.981617 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:06:56.982709 (+ 1092us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:06:56.982796 (+ 87us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:06:56.984192 (+ 1396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:06:56.985769 (+ 1577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:06:56.986423 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:06:56.987024 (+ 601us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:06:56.989327 (+ 2303us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:06:56.989333 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:06:56.989333 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:06:56.989348 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:06:56.991476 (+ 2128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:06:56.992006 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:06:56.992009 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:06:56.992010 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:06:56.992133 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:06:56.992482 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:06:56.992483 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:06:56.992483 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:06:56.992783 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:06:56.992855 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0812 21:06:56.993224 (+ 369us) server_negotiation.cc:299] Negotiation successful
0812 21:06:56.993378 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5339,"thread_start_us":97,"threads_started":1}
I20260812 21:06:56.994721 7088 heartbeater.cc:347] Connected to a master server at 127.11.254.190:38123
I20260812 21:06:56.994830 7088 heartbeater.cc:464] Registering TS with master...
I20260812 21:06:56.995034 7088 heartbeater.cc:511] Master 127.11.254.190:38123 requested a full tablet report, sending...
I20260812 21:06:56.995811 6794 ts_manager.cc:194] Registered new tserver with Master: 813f1c245a4e49c499d584012bf01a52 (127.11.254.129:38685)
I20260812 21:06:56.997040 6794 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.254.129@KRBTEST.COM'} at 127.11.254.129:55161
I20260812 21:06:56.998237 7088 heartbeater.cc:503] Master 127.11.254.190:38123 was elected leader, sending a full tablet report...
Aug 12 21:06:56 dist-test-slave-vpvm krb5kdc[5864](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568816, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.190@KRBTEST.COM for kudu/127.11.254.129@KRBTEST.COM
I20260812 21:06:57.006803 7102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:56.996746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51016 (local address 127.11.254.129:38685)
0812 21:06:56.997139 (+ 393us) server_negotiation.cc:206] Beginning negotiation
0812 21:06:56.997143 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 21:06:56.997152 (+ 9us) server_negotiation.cc:407] Connection header received
0812 21:06:56.997185 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:06:56.997188 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:06:56.997202 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:06:56.997257 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:06:56.998899 (+ 1642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:06:57.000053 (+ 1154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:06:57.001168 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:06:57.001631 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:06:57.001702 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:06:57.003951 (+ 2249us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:06:57.003959 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:06:57.003959 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:06:57.003966 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:06:57.005494 (+ 1528us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:06:57.005918 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:06:57.005920 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:06:57.005921 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:06:57.006023 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:06:57.006208 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:06:57.006208 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:06:57.006209 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:06:57.006350 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:06:57.006431 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0812 21:06:57.006646 (+ 215us) server_negotiation.cc:299] Negotiation successful
0812 21:06:57.006731 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":363,"thread_start_us":78,"threads_started":1}
I20260812 21:06:57.006842 7100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:56.996465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.129:38685 (local address 127.0.0.1:51016)
0812 21:06:56.996910 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0812 21:06:56.996947 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0812 21:06:56.997012 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:06:56.998059 (+ 1047us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:06:56.998067 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:06:56.998471 (+ 404us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:06:56.998537 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:06:56.998541 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:06:57.000136 (+ 1595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:06:57.000137 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:06:57.001082 (+ 945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:06:57.001085 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:06:57.001710 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:06:57.001711 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:06:57.001775 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:06:57.002253 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:06:57.002259 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:06:57.003857 (+ 1598us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:06:57.005585 (+ 1728us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:06:57.005592 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:06:57.005594 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:06:57.005846 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:06:57.006077 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:06:57.006078 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:06:57.006079 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:06:57.006149 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:06:57.006459 (+ 310us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:06:57.006466 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:06:57.006527 (+ 61us) client_negotiation.cc:769] Sending connection context
0812 21:06:57.006611 (+ 84us) client_negotiation.cc:240] Negotiation successful
0812 21:06:57.006744 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":415,"thread_start_us":111,"threads_started":1}
I20260812 21:06:57.007385 6990 tablet_service.cc:1514] Processing DeleteTablet for tablet 834b8470a47c4ee6a83492d58fdc4f51 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:06:36 UTC) from {username='kudu', principal='kudu/127.11.254.190@KRBTEST.COM'} at 127.0.0.1:51016
I20260812 21:06:57.007975 7103 tablet_replica.cc:330] stopping tablet replica
I20260812 21:06:57.008090 7103 raft_consensus.cc:2229] T 834b8470a47c4ee6a83492d58fdc4f51 P 813f1c245a4e49c499d584012bf01a52 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 21:06:57.008163 7103 raft_consensus.cc:2258] T 834b8470a47c4ee6a83492d58fdc4f51 P 813f1c245a4e49c499d584012bf01a52 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 21:06:57.008496 7103 ts_tablet_manager.cc:1905] T 834b8470a47c4ee6a83492d58fdc4f51 P 813f1c245a4e49c499d584012bf01a52: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 21:06:57.011240 7103 ts_tablet_manager.cc:1918] T 834b8470a47c4ee6a83492d58fdc4f51 P 813f1c245a4e49c499d584012bf01a52: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 21:06:57.011322 7103 log.cc:1196] T 834b8470a47c4ee6a83492d58fdc4f51 P 813f1c245a4e49c499d584012bf01a52: Deleting WAL directory at /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/ts-0/wal/wals/834b8470a47c4ee6a83492d58fdc4f51
I20260812 21:06:57.011554 7103 ts_tablet_manager.cc:1939] T 834b8470a47c4ee6a83492d58fdc4f51 P 813f1c245a4e49c499d584012bf01a52: Deleting consensus metadata
I20260812 21:06:57.012219 6766 catalog_manager.cc:4841] TS 813f1c245a4e49c499d584012bf01a52 (127.11.254.129:38685): tablet 834b8470a47c4ee6a83492d58fdc4f51 (table test-table [id=6bd6767f29e84eafb7d9ee9d904f5b0b]) successfully deleted
W20260812 21:06:57.070503 7096 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:06:57.070719 7096 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:06:57.070742 7096 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:06:57.071921 7096 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:06:57.071975 7096 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:06:57.072002 7096 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.254.130
I20260812 21:06:57.073460 7096 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.254.148:34711
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.130.keytab
--principal=kudu/127.11.254.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.254.130:39975
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.254.130
--webserver_port=43347
--tserver_master_addrs=127.11.254.190:38123
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7096
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.254.130
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:06:57.073709 7096 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:06:57.073938 7096 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:06:57.079064 7113 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:06:57.079064 7119 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:06:57.079118 7115 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:06:57.079265 7096 server_base.cc:1034] running on GCE node
I20260812 21:06:57.079528 7096 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:06:57.079751 7096 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:06:57.080892 7096 hybrid_clock.cc:648] HybridClock initialized: now 1786568817080869 us; error 38 us; skew 500 ppm
Aug 12 21:06:57 dist-test-slave-vpvm krb5kdc[5864](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568817, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:06:57.083452 7096 init.cc:377] Logged in from keytab as kudu/127.11.254.130@KRBTEST.COM (short username kudu)
I20260812 21:06:57.083910 7096 webserver.cc:460] Webserver started at http://127.11.254.130:43347/ using document root <none> and password file <none>
I20260812 21:06:57.084110 7096 fs_manager.cc:359] Metadata directory not provided
I20260812 21:06:57.084162 7096 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:06:57.085491 7096 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 21:06:57.086097 7128 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:06:57.086256 7096 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 21:06:57.086331 7096 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/ts-1/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/ts-1/wal
uuid: "8eb6c52ef06a4d5f97762c59705d0047"
format_stamp: "Formatted at 2026-08-12 21:06:36 on dist-test-slave-vpvm"
server_key: "ec49e5db1cd29defa3404affc9b1cc98"
server_key_iv: "bc96b50eff0bf064cfb099c763c04d06"
server_key_version: "encryptionkey@0"
I20260812 21:06:57.086714 7096 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:06:57.099982 7096 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:06:57.102188 7096 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:06:57.102334 7096 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:06:57.102559 7096 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:06:57.102854 7096 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:06:57.102888 7096 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:06:57.102917 7096 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:06:57.102941 7096 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:06:57.108574 7096 rpc_server.cc:307] RPC server started. Bound to: 127.11.254.130:39975
I20260812 21:06:57.108637 7311 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.254.130:39975 every 8 connection(s)
I20260812 21:06:57.108963 7096 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/ts-1/data/info.pb
Aug 12 21:06:57 dist-test-slave-vpvm krb5kdc[5864](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568817, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.130@KRBTEST.COM for kudu/127.11.254.190@KRBTEST.COM
I20260812 21:06:57.115319 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 7096
I20260812 21:06:57.115674 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.254.131:38309
--local_ip_for_outbound_sockets=127.11.254.131
--tserver_master_addrs=127.11.254.190:38123
--webserver_port=46091
--webserver_interface=127.11.254.131
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.131.keytab
--principal=kudu/127.11.254.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.254.148:34711
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 21:06:57.118387 7094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:57.109668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.130:52145 (local address 127.11.254.190:38123)
0812 21:06:57.109729 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0812 21:06:57.109731 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:06:57.109945 (+ 214us) server_negotiation.cc:407] Connection header received
0812 21:06:57.110497 (+ 552us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:06:57.110499 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:06:57.110511 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:06:57.110571 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:06:57.111523 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:06:57.112024 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:06:57.113085 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:06:57.113294 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:06:57.113351 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:06:57.115575 (+ 2224us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:06:57.115577 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:06:57.115578 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:06:57.115587 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:06:57.117005 (+ 1418us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:06:57.117403 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:06:57.117404 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:06:57.117404 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:06:57.117459 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:06:57.117695 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:06:57.117696 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:06:57.117696 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:06:57.117814 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:06:57.117871 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0812 21:06:57.118194 (+ 323us) server_negotiation.cc:299] Negotiation successful
0812 21:06:57.118295 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260812 21:06:57.118566 7316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:57.109660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:38123 (local address 127.11.254.130:52145)
0812 21:06:57.109827 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0812 21:06:57.109864 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0812 21:06:57.110392 (+ 528us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:06:57.110646 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:06:57.110650 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:06:57.111222 (+ 572us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:06:57.111422 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:06:57.111427 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:06:57.112119 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:06:57.112120 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:06:57.112996 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:06:57.112999 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:06:57.113368 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:06:57.113369 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:06:57.113496 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:06:57.114017 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:06:57.114026 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:06:57.115478 (+ 1452us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:06:57.117091 (+ 1613us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:06:57.117101 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:06:57.117103 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:06:57.117331 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:06:57.117515 (+ 184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:06:57.117516 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:06:57.117517 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:06:57.117636 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:06:57.117869 (+ 233us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:06:57.117877 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:06:57.118069 (+ 192us) client_negotiation.cc:769] Sending connection context
0812 21:06:57.118164 (+ 95us) client_negotiation.cc:240] Negotiation successful
0812 21:06:57.118331 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":77,"threads_started":1}
I20260812 21:06:57.119033 7312 heartbeater.cc:347] Connected to a master server at 127.11.254.190:38123
I20260812 21:06:57.119134 7312 heartbeater.cc:464] Registering TS with master...
I20260812 21:06:57.119313 7312 heartbeater.cc:511] Master 127.11.254.190:38123 requested a full tablet report, sending...
I20260812 21:06:57.119699 6794 ts_manager.cc:194] Registered new tserver with Master: 8eb6c52ef06a4d5f97762c59705d0047 (127.11.254.130:39975)
I20260812 21:06:57.120204 6794 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.254.130@KRBTEST.COM'} at 127.11.254.130:52145
W20260812 21:06:57.200531 7317 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:06:57.200757 7317 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:06:57.200783 7317 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:06:57.201917 7317 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:06:57.201978 7317 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:06:57.202016 7317 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.254.131
I20260812 21:06:57.203488 7317 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.254.148:34711
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.131.keytab
--principal=kudu/127.11.254.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.254.131:38309
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.254.131
--webserver_port=46091
--tserver_master_addrs=127.11.254.190:38123
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7317
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.254.131
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:06:57.203681 7317 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:06:57.203873 7317 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:06:57.208995 7331 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:06:57.209007 7329 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:06:57.209003 7335 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:06:57.209111 7317 server_base.cc:1034] running on GCE node
I20260812 21:06:57.209446 7317 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:06:57.209722 7317 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:06:57.210897 7317 hybrid_clock.cc:648] HybridClock initialized: now 1786568817210881 us; error 31 us; skew 500 ppm
Aug 12 21:06:57 dist-test-slave-vpvm krb5kdc[5864](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568817, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:06:57.213550 7317 init.cc:377] Logged in from keytab as kudu/127.11.254.131@KRBTEST.COM (short username kudu)
I20260812 21:06:57.214080 7317 webserver.cc:460] Webserver started at http://127.11.254.131:46091/ using document root <none> and password file <none>
I20260812 21:06:57.214279 7317 fs_manager.cc:359] Metadata directory not provided
I20260812 21:06:57.214334 7317 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:06:57.215775 7317 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 21:06:57.216390 7346 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:06:57.216591 7317 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 21:06:57.216674 7317 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/ts-2/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/ts-2/wal
uuid: "024bc23e991c46bc813212a50b07f065"
format_stamp: "Formatted at 2026-08-12 21:06:36 on dist-test-slave-vpvm"
server_key: "7a0d4318ad00daca63381859cea667cd"
server_key_iv: "319262ae1c58b20aebe38eb1be4a839d"
server_key_version: "encryptionkey@0"
I20260812 21:06:57.217046 7317 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:06:57.227442 7317 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:06:57.229616 7317 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:06:57.229758 7317 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:06:57.230019 7317 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:06:57.230342 7317 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:06:57.230377 7317 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:06:57.230410 7317 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:06:57.230434 7317 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:06:57.236181 7317 rpc_server.cc:307] RPC server started. Bound to: 127.11.254.131:38309
I20260812 21:06:57.236238 7539 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.254.131:38309 every 8 connection(s)
I20260812 21:06:57.236572 7317 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/ts-2/data/info.pb
I20260812 21:06:57.242180 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 7317
Aug 12 21:06:57 dist-test-slave-vpvm krb5kdc[5864](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568817, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.131@KRBTEST.COM for kudu/127.11.254.190@KRBTEST.COM
I20260812 21:06:57.245518 7094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:57.237265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.131:55297 (local address 127.11.254.190:38123)
0812 21:06:57.237336 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0812 21:06:57.237338 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:06:57.237557 (+ 219us) server_negotiation.cc:407] Connection header received
0812 21:06:57.238078 (+ 521us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:06:57.238079 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:06:57.238090 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:06:57.238141 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:06:57.239109 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:06:57.239620 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:06:57.240583 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:06:57.240780 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:06:57.240830 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:06:57.243008 (+ 2178us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:06:57.243010 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:06:57.243010 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:06:57.243017 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:06:57.244263 (+ 1246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:06:57.244629 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:06:57.244630 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:06:57.244630 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:06:57.244684 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:06:57.244906 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:06:57.244907 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:06:57.244907 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:06:57.245015 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:06:57.245071 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0812 21:06:57.245339 (+ 268us) server_negotiation.cc:299] Negotiation successful
0812 21:06:57.245429 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 21:06:57.245692 7544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:57.237262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:38123 (local address 127.11.254.131:55297)
0812 21:06:57.237411 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0812 21:06:57.237449 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 21:06:57.237982 (+ 533us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:06:57.238199 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:06:57.238202 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:06:57.238797 (+ 595us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:06:57.238998 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:06:57.239003 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:06:57.239697 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:06:57.239698 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:06:57.240493 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:06:57.240496 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:06:57.240839 (+ 343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:06:57.240839 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:06:57.240936 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:06:57.241421 (+ 485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:06:57.241430 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:06:57.242914 (+ 1484us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:06:57.244338 (+ 1424us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:06:57.244345 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:06:57.244347 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:06:57.244562 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:06:57.244739 (+ 177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:06:57.244740 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:06:57.244740 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:06:57.244847 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:06:57.245078 (+ 231us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:06:57.245084 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:06:57.245220 (+ 136us) client_negotiation.cc:769] Sending connection context
0812 21:06:57.245309 (+ 89us) client_negotiation.cc:240] Negotiation successful
0812 21:06:57.245470 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":80,"threads_started":1}
I20260812 21:06:57.246179 7541 heartbeater.cc:347] Connected to a master server at 127.11.254.190:38123
I20260812 21:06:57.246292 7541 heartbeater.cc:464] Registering TS with master...
I20260812 21:06:57.246500 7541 heartbeater.cc:511] Master 127.11.254.190:38123 requested a full tablet report, sending...
I20260812 21:06:57.246906 6794 ts_manager.cc:194] Registered new tserver with Master: 024bc23e991c46bc813212a50b07f065 (127.11.254.131:38309)
I20260812 21:06:57.247262 6794 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.254.131@KRBTEST.COM'} at 127.11.254.131:55297
I20260812 21:06:57.249033 7547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:57.242612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53196 (local address 127.11.254.190:38123)
0812 21:06:57.242752 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0812 21:06:57.242754 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:06:57.242824 (+ 70us) server_negotiation.cc:407] Connection header received
0812 21:06:57.242892 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:06:57.242893 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:06:57.242904 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:06:57.242962 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:06:57.243546 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:06:57.244025 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:06:57.244748 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:06:57.244946 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:06:57.244999 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:06:57.246433 (+ 1434us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:06:57.246434 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:06:57.246434 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:06:57.246440 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:06:57.247936 (+ 1496us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:06:57.248297 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:06:57.248298 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:06:57.248299 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:06:57.248357 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:06:57.248523 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:06:57.248524 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:06:57.248525 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:06:57.248635 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:06:57.248810 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0812 21:06:57.248849 (+ 39us) server_negotiation.cc:299] Negotiation successful
0812 21:06:57.248931 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":54,"threads_started":1}
I20260812 21:06:57.249666 12282 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 21:06:57.252115 7547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:57.249982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53210 (local address 127.11.254.190:38123)
0812 21:06:57.250027 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0812 21:06:57.250029 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:06:57.250163 (+ 134us) server_negotiation.cc:407] Connection header received
0812 21:06:57.250226 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:06:57.250227 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:06:57.250255 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:06:57.250301 (+ 46us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 21:06:57.250384 (+ 83us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:06:57.250889 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:06:57.251598 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:06:57.251779 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:06:57.251821 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:06:57.251928 (+ 107us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 21:06:57.251965 (+ 37us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0812 21:06:57.252072 (+ 107us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:53210: token expired
Metrics: {"server-negotiator.queue_time_us":24}
W20260812 21:06:57.252156 7547 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:53210: token expired
W20260812 21:06:57.252312 6681 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.11.254.190:38123)
I20260812 21:06:57.257825 7547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:57.252540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53220 (local address 127.11.254.190:38123)
0812 21:06:57.252583 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0812 21:06:57.252585 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:06:57.252593 (+ 8us) server_negotiation.cc:407] Connection header received
0812 21:06:57.252640 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:06:57.252641 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:06:57.252649 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:06:57.252697 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:06:57.253130 (+ 433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:06:57.253569 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:06:57.254272 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:06:57.254458 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:06:57.254514 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:06:57.255544 (+ 1030us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:06:57.255546 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:06:57.255546 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:06:57.255554 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:06:57.256836 (+ 1282us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:06:57.257146 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:06:57.257148 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:06:57.257148 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:06:57.257203 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:06:57.257364 (+ 161us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:06:57.257365 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:06:57.257365 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:06:57.257470 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:06:57.257574 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0812 21:06:57.257652 (+ 78us) server_negotiation.cc:299] Negotiation successful
0812 21:06:57.257727 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 21:06:57.258992 6794 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53220:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 21:06:57.260648 6794 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 21:06:57.263007 6990 tablet_service.cc:1467] Processing CreateTablet for tablet d02b46993aa243c0a74732ae7ddd2db6 (DEFAULT_TABLE table=test-table [id=9f8260ba21404588b1419dfa4639b806]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:06:57.263226 6990 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d02b46993aa243c0a74732ae7ddd2db6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:06:57.264490 6919 tablet_bootstrap.cc:492] T d02b46993aa243c0a74732ae7ddd2db6 P 813f1c245a4e49c499d584012bf01a52: Bootstrap starting.
I20260812 21:06:57.264825 6919 tablet_bootstrap.cc:654] T d02b46993aa243c0a74732ae7ddd2db6 P 813f1c245a4e49c499d584012bf01a52: Neither blocks nor log segments found. Creating new log.
I20260812 21:06:57.265744 6919 tablet_bootstrap.cc:492] T d02b46993aa243c0a74732ae7ddd2db6 P 813f1c245a4e49c499d584012bf01a52: No bootstrap required, opened a new log
I20260812 21:06:57.265805 6919 ts_tablet_manager.cc:1397] T d02b46993aa243c0a74732ae7ddd2db6 P 813f1c245a4e49c499d584012bf01a52: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260812 21:06:57.265996 6919 raft_consensus.cc:348] T d02b46993aa243c0a74732ae7ddd2db6 P 813f1c245a4e49c499d584012bf01a52 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "024bc23e991c46bc813212a50b07f065" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 38309 } } peers { permanent_uuid: "8eb6c52ef06a4d5f97762c59705d0047" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 39975 } } peers { permanent_uuid: "813f1c245a4e49c499d584012bf01a52" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 38685 } }
I20260812 21:06:57.266093 6919 raft_consensus.cc:374] T d02b46993aa243c0a74732ae7ddd2db6 P 813f1c245a4e49c499d584012bf01a52 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:06:57.266151 6919 raft_consensus.cc:729] T d02b46993aa243c0a74732ae7ddd2db6 P 813f1c245a4e49c499d584012bf01a52 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 813f1c245a4e49c499d584012bf01a52, State: Initialized, Role: FOLLOWER
I20260812 21:06:57.266196 6919 consensus_queue.cc:260] T d02b46993aa243c0a74732ae7ddd2db6 P 813f1c245a4e49c499d584012bf01a52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "024bc23e991c46bc813212a50b07f065" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 38309 } } peers { permanent_uuid: "8eb6c52ef06a4d5f97762c59705d0047" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 39975 } } peers { permanent_uuid: "813f1c245a4e49c499d584012bf01a52" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 38685 } }
I20260812 21:06:57.266275 6919 ts_tablet_manager.cc:1428] T d02b46993aa243c0a74732ae7ddd2db6 P 813f1c245a4e49c499d584012bf01a52: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260812 21:06:57.266759 7100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:57.262555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.130:39975 (local address 127.0.0.1:49548)
0812 21:06:57.262608 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0812 21:06:57.262619 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 21:06:57.262666 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:06:57.263423 (+ 757us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:06:57.263424 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:06:57.263427 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:06:57.263474 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:06:57.263475 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:06:57.264693 (+ 1218us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:06:57.264694 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:06:57.265533 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:06:57.265536 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:06:57.266485 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:06:57.266486 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:06:57.266562 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:06:57.266585 (+ 23us) client_negotiation.cc:769] Sending connection context
0812 21:06:57.266642 (+ 57us) client_negotiation.cc:240] Negotiation successful
0812 21:06:57.266694 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260812 21:06:57.266919 7551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:57.262555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.131:38309 (local address 127.0.0.1:35394)
0812 21:06:57.262773 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0812 21:06:57.262782 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 21:06:57.262821 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:06:57.263106 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:06:57.263107 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:06:57.263110 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:06:57.263179 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:06:57.263181 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:06:57.264582 (+ 1401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:06:57.264583 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:06:57.265772 (+ 1189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:06:57.265776 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:06:57.266693 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:06:57.266694 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:06:57.266760 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:06:57.266764 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 21:06:57.266808 (+ 44us) client_negotiation.cc:240] Negotiation successful
0812 21:06:57.266848 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":202,"mutex_wait_us":51,"thread_start_us":94,"threads_started":1}
I20260812 21:06:57.266965 7554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:57.262755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49548 (local address 127.11.254.130:39975)
0812 21:06:57.263265 (+ 510us) server_negotiation.cc:206] Beginning negotiation
0812 21:06:57.263269 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 21:06:57.263279 (+ 10us) server_negotiation.cc:407] Connection header received
0812 21:06:57.263316 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:06:57.263318 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:06:57.263333 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:06:57.263474 (+ 141us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:06:57.263571 (+ 97us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:06:57.264611 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:06:57.265636 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:06:57.266366 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:06:57.266485 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:06:57.266555 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 21:06:57.266839 (+ 284us) server_negotiation.cc:299] Negotiation successful
0812 21:06:57.266890 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":466,"thread_start_us":90,"threads_started":1}
I20260812 21:06:57.267249 7557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:57.262742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35394 (local address 127.11.254.131:38309)
0812 21:06:57.262945 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0812 21:06:57.262950 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 21:06:57.262959 (+ 9us) server_negotiation.cc:407] Connection header received
0812 21:06:57.262997 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:06:57.262999 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:06:57.263011 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:06:57.263089 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:06:57.263263 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:06:57.264485 (+ 1222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:06:57.265875 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:06:57.266621 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:06:57.266685 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:06:57.266738 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0812 21:06:57.267119 (+ 381us) server_negotiation.cc:299] Negotiation successful
0812 21:06:57.267169 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":114,"threads_started":1}
I20260812 21:06:57.267655 7204 tablet_service.cc:1467] Processing CreateTablet for tablet d02b46993aa243c0a74732ae7ddd2db6 (DEFAULT_TABLE table=test-table [id=9f8260ba21404588b1419dfa4639b806]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:06:57.268052 7204 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d02b46993aa243c0a74732ae7ddd2db6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:06:57.267992 7428 tablet_service.cc:1467] Processing CreateTablet for tablet d02b46993aa243c0a74732ae7ddd2db6 (DEFAULT_TABLE table=test-table [id=9f8260ba21404588b1419dfa4639b806]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:06:57.268307 7428 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d02b46993aa243c0a74732ae7ddd2db6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:06:57.271801 7560 tablet_bootstrap.cc:492] T d02b46993aa243c0a74732ae7ddd2db6 P 024bc23e991c46bc813212a50b07f065: Bootstrap starting.
I20260812 21:06:57.271997 7559 tablet_bootstrap.cc:492] T d02b46993aa243c0a74732ae7ddd2db6 P 8eb6c52ef06a4d5f97762c59705d0047: Bootstrap starting.
I20260812 21:06:57.272564 7560 tablet_bootstrap.cc:654] T d02b46993aa243c0a74732ae7ddd2db6 P 024bc23e991c46bc813212a50b07f065: Neither blocks nor log segments found. Creating new log.
I20260812 21:06:57.272634 7559 tablet_bootstrap.cc:654] T d02b46993aa243c0a74732ae7ddd2db6 P 8eb6c52ef06a4d5f97762c59705d0047: Neither blocks nor log segments found. Creating new log.
I20260812 21:06:57.273195 7560 log.cc:824] T d02b46993aa243c0a74732ae7ddd2db6 P 024bc23e991c46bc813212a50b07f065: Log is configured to *not* fsync() on all Append() calls
I20260812 21:06:57.273195 7559 log.cc:824] T d02b46993aa243c0a74732ae7ddd2db6 P 8eb6c52ef06a4d5f97762c59705d0047: Log is configured to *not* fsync() on all Append() calls
I20260812 21:06:57.274005 7559 tablet_bootstrap.cc:492] T d02b46993aa243c0a74732ae7ddd2db6 P 8eb6c52ef06a4d5f97762c59705d0047: No bootstrap required, opened a new log
I20260812 21:06:57.274082 7559 ts_tablet_manager.cc:1397] T d02b46993aa243c0a74732ae7ddd2db6 P 8eb6c52ef06a4d5f97762c59705d0047: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 21:06:57.274206 7560 tablet_bootstrap.cc:492] T d02b46993aa243c0a74732ae7ddd2db6 P 024bc23e991c46bc813212a50b07f065: No bootstrap required, opened a new log
I20260812 21:06:57.274271 7560 ts_tablet_manager.cc:1397] T d02b46993aa243c0a74732ae7ddd2db6 P 024bc23e991c46bc813212a50b07f065: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 21:06:57.276075 7559 raft_consensus.cc:348] T d02b46993aa243c0a74732ae7ddd2db6 P 8eb6c52ef06a4d5f97762c59705d0047 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "024bc23e991c46bc813212a50b07f065" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 38309 } } peers { permanent_uuid: "8eb6c52ef06a4d5f97762c59705d0047" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 39975 } } peers { permanent_uuid: "813f1c245a4e49c499d584012bf01a52" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 38685 } }
I20260812 21:06:57.276122 7560 raft_consensus.cc:348] T d02b46993aa243c0a74732ae7ddd2db6 P 024bc23e991c46bc813212a50b07f065 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "024bc23e991c46bc813212a50b07f065" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 38309 } } peers { permanent_uuid: "8eb6c52ef06a4d5f97762c59705d0047" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 39975 } } peers { permanent_uuid: "813f1c245a4e49c499d584012bf01a52" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 38685 } }
I20260812 21:06:57.276278 7559 raft_consensus.cc:374] T d02b46993aa243c0a74732ae7ddd2db6 P 8eb6c52ef06a4d5f97762c59705d0047 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:06:57.276278 7560 raft_consensus.cc:374] T d02b46993aa243c0a74732ae7ddd2db6 P 024bc23e991c46bc813212a50b07f065 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:06:57.276314 7559 raft_consensus.cc:729] T d02b46993aa243c0a74732ae7ddd2db6 P 8eb6c52ef06a4d5f97762c59705d0047 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8eb6c52ef06a4d5f97762c59705d0047, State: Initialized, Role: FOLLOWER
I20260812 21:06:57.276314 7560 raft_consensus.cc:729] T d02b46993aa243c0a74732ae7ddd2db6 P 024bc23e991c46bc813212a50b07f065 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 024bc23e991c46bc813212a50b07f065, State: Initialized, Role: FOLLOWER
I20260812 21:06:57.276467 7560 consensus_queue.cc:260] T d02b46993aa243c0a74732ae7ddd2db6 P 024bc23e991c46bc813212a50b07f065 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "024bc23e991c46bc813212a50b07f065" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 38309 } } peers { permanent_uuid: "8eb6c52ef06a4d5f97762c59705d0047" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 39975 } } peers { permanent_uuid: "813f1c245a4e49c499d584012bf01a52" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 38685 } }
I20260812 21:06:57.276467 7559 consensus_queue.cc:260] T d02b46993aa243c0a74732ae7ddd2db6 P 8eb6c52ef06a4d5f97762c59705d0047 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "024bc23e991c46bc813212a50b07f065" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 38309 } } peers { permanent_uuid: "8eb6c52ef06a4d5f97762c59705d0047" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 39975 } } peers { permanent_uuid: "813f1c245a4e49c499d584012bf01a52" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 38685 } }
I20260812 21:06:57.276767 7559 ts_tablet_manager.cc:1428] T d02b46993aa243c0a74732ae7ddd2db6 P 8eb6c52ef06a4d5f97762c59705d0047: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 21:06:57.276767 7560 ts_tablet_manager.cc:1428] T d02b46993aa243c0a74732ae7ddd2db6 P 024bc23e991c46bc813212a50b07f065: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260812 21:06:57.276901 7312 heartbeater.cc:503] Master 127.11.254.190:38123 was elected leader, sending a full tablet report...
I20260812 21:06:57.277164 7541 heartbeater.cc:503] Master 127.11.254.190:38123 was elected leader, sending a full tablet report...
I20260812 21:06:57.351986 7564 raft_consensus.cc:482] T d02b46993aa243c0a74732ae7ddd2db6 P 024bc23e991c46bc813212a50b07f065 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 21:06:57.352128 7564 raft_consensus.cc:504] T d02b46993aa243c0a74732ae7ddd2db6 P 024bc23e991c46bc813212a50b07f065 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "024bc23e991c46bc813212a50b07f065" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 38309 } } peers { permanent_uuid: "8eb6c52ef06a4d5f97762c59705d0047" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 39975 } } peers { permanent_uuid: "813f1c245a4e49c499d584012bf01a52" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 38685 } }
I20260812 21:06:57.352514 7564 leader_election.cc:288] T d02b46993aa243c0a74732ae7ddd2db6 P 024bc23e991c46bc813212a50b07f065 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8eb6c52ef06a4d5f97762c59705d0047 (127.11.254.130:39975), 813f1c245a4e49c499d584012bf01a52 (127.11.254.129:38685)
I20260812 21:06:57.356389 7554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:57.352812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.131:45777 (local address 127.11.254.130:39975)
0812 21:06:57.352867 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0812 21:06:57.352869 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:06:57.352878 (+ 9us) server_negotiation.cc:407] Connection header received
0812 21:06:57.352908 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:06:57.352909 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:06:57.352917 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:06:57.352969 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:06:57.353138 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:06:57.353880 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:06:57.355314 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:06:57.355991 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:06:57.356071 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:06:57.356081 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0812 21:06:57.356287 (+ 206us) server_negotiation.cc:299] Negotiation successful
0812 21:06:57.356332 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260812 21:06:57.356388 7544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:57.352710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.130:39975 (local address 127.11.254.131:45777)
0812 21:06:57.352761 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0812 21:06:57.352773 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 21:06:57.352825 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:06:57.352993 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:06:57.352994 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:06:57.352996 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:06:57.353072 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:06:57.353074 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:06:57.353990 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:06:57.353991 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:06:57.355206 (+ 1215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:06:57.355209 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:06:57.356088 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:06:57.356090 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:06:57.356184 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:06:57.356186 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 21:06:57.356245 (+ 59us) client_negotiation.cc:240] Negotiation successful
0812 21:06:57.356283 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27,"mutex_wait_us":2}
I20260812 21:06:57.356396 7567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:57.352719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.129:38685 (local address 127.11.254.131:42981)
0812 21:06:57.352934 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0812 21:06:57.352943 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 21:06:57.352992 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:06:57.353127 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:06:57.353127 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:06:57.353129 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:06:57.353191 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:06:57.353192 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:06:57.354091 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:06:57.354092 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:06:57.355207 (+ 1115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:06:57.355209 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:06:57.356088 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:06:57.356090 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:06:57.356181 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:06:57.356186 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 21:06:57.356234 (+ 48us) client_negotiation.cc:240] Negotiation successful
0812 21:06:57.356283 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":199,"mutex_wait_us":65,"thread_start_us":112,"threads_started":1}
I20260812 21:06:57.356374 7102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:57.352803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.131:42981 (local address 127.11.254.129:38685)
0812 21:06:57.352867 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0812 21:06:57.352869 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:06:57.352998 (+ 129us) server_negotiation.cc:407] Connection header received
0812 21:06:57.353035 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:06:57.353035 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:06:57.353061 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:06:57.353116 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:06:57.353248 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:06:57.354028 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:06:57.355314 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:06:57.355991 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:06:57.356071 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:06:57.356210 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0812 21:06:57.356243 (+ 33us) server_negotiation.cc:299] Negotiation successful
0812 21:06:57.356298 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260812 21:06:57.356774 7238 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d02b46993aa243c0a74732ae7ddd2db6" candidate_uuid: "024bc23e991c46bc813212a50b07f065" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8eb6c52ef06a4d5f97762c59705d0047" is_pre_election: true
I20260812 21:06:57.356806 7021 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d02b46993aa243c0a74732ae7ddd2db6" candidate_uuid: "024bc23e991c46bc813212a50b07f065" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "813f1c245a4e49c499d584012bf01a52" is_pre_election: true
I20260812 21:06:57.356930 7238 raft_consensus.cc:2454] T d02b46993aa243c0a74732ae7ddd2db6 P 8eb6c52ef06a4d5f97762c59705d0047 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 024bc23e991c46bc813212a50b07f065 in term 0.
I20260812 21:06:57.356943 7021 raft_consensus.cc:2454] T d02b46993aa243c0a74732ae7ddd2db6 P 813f1c245a4e49c499d584012bf01a52 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 024bc23e991c46bc813212a50b07f065 in term 0.
I20260812 21:06:57.357161 7348 leader_election.cc:302] T d02b46993aa243c0a74732ae7ddd2db6 P 024bc23e991c46bc813212a50b07f065 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 024bc23e991c46bc813212a50b07f065, 813f1c245a4e49c499d584012bf01a52; no voters:
I20260812 21:06:57.357295 7564 raft_consensus.cc:2781] T d02b46993aa243c0a74732ae7ddd2db6 P 024bc23e991c46bc813212a50b07f065 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 21:06:57.357352 7564 raft_consensus.cc:482] T d02b46993aa243c0a74732ae7ddd2db6 P 024bc23e991c46bc813212a50b07f065 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 21:06:57.357374 7564 raft_consensus.cc:3037] T d02b46993aa243c0a74732ae7ddd2db6 P 024bc23e991c46bc813212a50b07f065 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:06:57.358294 7564 raft_consensus.cc:504] T d02b46993aa243c0a74732ae7ddd2db6 P 024bc23e991c46bc813212a50b07f065 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "024bc23e991c46bc813212a50b07f065" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 38309 } } peers { permanent_uuid: "8eb6c52ef06a4d5f97762c59705d0047" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 39975 } } peers { permanent_uuid: "813f1c245a4e49c499d584012bf01a52" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 38685 } }
I20260812 21:06:57.358429 7564 leader_election.cc:288] T d02b46993aa243c0a74732ae7ddd2db6 P 024bc23e991c46bc813212a50b07f065 [CANDIDATE]: Term 1 election: Requested vote from peers 8eb6c52ef06a4d5f97762c59705d0047 (127.11.254.130:39975), 813f1c245a4e49c499d584012bf01a52 (127.11.254.129:38685)
I20260812 21:06:57.358604 7021 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d02b46993aa243c0a74732ae7ddd2db6" candidate_uuid: "024bc23e991c46bc813212a50b07f065" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "813f1c245a4e49c499d584012bf01a52"
I20260812 21:06:57.358637 7238 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d02b46993aa243c0a74732ae7ddd2db6" candidate_uuid: "024bc23e991c46bc813212a50b07f065" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8eb6c52ef06a4d5f97762c59705d0047"
I20260812 21:06:57.358693 7021 raft_consensus.cc:3037] T d02b46993aa243c0a74732ae7ddd2db6 P 813f1c245a4e49c499d584012bf01a52 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:06:57.358736 7238 raft_consensus.cc:3037] T d02b46993aa243c0a74732ae7ddd2db6 P 8eb6c52ef06a4d5f97762c59705d0047 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:06:57.359411 7021 raft_consensus.cc:2454] T d02b46993aa243c0a74732ae7ddd2db6 P 813f1c245a4e49c499d584012bf01a52 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 024bc23e991c46bc813212a50b07f065 in term 1.
I20260812 21:06:57.359501 7238 raft_consensus.cc:2454] T d02b46993aa243c0a74732ae7ddd2db6 P 8eb6c52ef06a4d5f97762c59705d0047 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 024bc23e991c46bc813212a50b07f065 in term 1.
W20260812 21:06:57.359503 7313 tablet.cc:2369] T d02b46993aa243c0a74732ae7ddd2db6 P 8eb6c52ef06a4d5f97762c59705d0047: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 21:06:57.359692 7348 leader_election.cc:302] T d02b46993aa243c0a74732ae7ddd2db6 P 024bc23e991c46bc813212a50b07f065 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 024bc23e991c46bc813212a50b07f065, 813f1c245a4e49c499d584012bf01a52; no voters:
I20260812 21:06:57.359838 7564 raft_consensus.cc:2781] T d02b46993aa243c0a74732ae7ddd2db6 P 024bc23e991c46bc813212a50b07f065 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:06:57.359984 7564 raft_consensus.cc:686] T d02b46993aa243c0a74732ae7ddd2db6 P 024bc23e991c46bc813212a50b07f065 [term 1 LEADER]: Becoming Leader. State: Replica: 024bc23e991c46bc813212a50b07f065, State: Running, Role: LEADER
I20260812 21:06:57.360069 7564 consensus_queue.cc:237] T d02b46993aa243c0a74732ae7ddd2db6 P 024bc23e991c46bc813212a50b07f065 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "024bc23e991c46bc813212a50b07f065" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 38309 } } peers { permanent_uuid: "8eb6c52ef06a4d5f97762c59705d0047" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 39975 } } peers { permanent_uuid: "813f1c245a4e49c499d584012bf01a52" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 38685 } }
I20260812 21:06:57.360918 6794 catalog_manager.cc:5495] T d02b46993aa243c0a74732ae7ddd2db6 P 024bc23e991c46bc813212a50b07f065 reported cstate change: term changed from 0 to 1, leader changed from <none> to 024bc23e991c46bc813212a50b07f065 (127.11.254.131). New cstate: current_term: 1 leader_uuid: "024bc23e991c46bc813212a50b07f065" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "024bc23e991c46bc813212a50b07f065" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 38309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8eb6c52ef06a4d5f97762c59705d0047" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 39975 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "813f1c245a4e49c499d584012bf01a52" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 38685 } health_report { overall_health: UNKNOWN } } }
I20260812 21:06:57.385301 7557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:06:57.383011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35408 (local address 127.11.254.131:38309)
0812 21:06:57.383074 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0812 21:06:57.383076 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:06:57.383084 (+ 8us) server_negotiation.cc:407] Connection header received
0812 21:06:57.383122 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:06:57.383123 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:06:57.383151 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:06:57.383210 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 21:06:57.383325 (+ 115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:06:57.383838 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:06:57.384586 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:06:57.384785 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:06:57.384839 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:06:57.384982 (+ 143us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 21:06:57.385089 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 21:06:57.385160 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0812 21:06:57.385198 (+ 38us) server_negotiation.cc:299] Negotiation successful
0812 21:06:57.385220 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260812 21:06:57.387282 7238 raft_consensus.cc:1261] T d02b46993aa243c0a74732ae7ddd2db6 P 8eb6c52ef06a4d5f97762c59705d0047 [term 1 FOLLOWER]: Refusing update from remote peer 024bc23e991c46bc813212a50b07f065: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 21:06:57.387302 7021 raft_consensus.cc:1261] T d02b46993aa243c0a74732ae7ddd2db6 P 813f1c245a4e49c499d584012bf01a52 [term 1 FOLLOWER]: Refusing update from remote peer 024bc23e991c46bc813212a50b07f065: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 21:06:57.387679 7569 consensus_queue.cc:1035] T d02b46993aa243c0a74732ae7ddd2db6 P 024bc23e991c46bc813212a50b07f065 [LEADER]: Connected to new peer: Peer: permanent_uuid: "813f1c245a4e49c499d584012bf01a52" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 38685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 21:06:57.387828 7564 consensus_queue.cc:1035] T d02b46993aa243c0a74732ae7ddd2db6 P 024bc23e991c46bc813212a50b07f065 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8eb6c52ef06a4d5f97762c59705d0047" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 39975 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 21:06:57.390487 7574 mvcc.cc:205] Tried to move back new op lower bound from 7317785876015439872 to 7317785875907547136. Current Snapshot: MvccSnapshot[applied={T|T < 7317785876015439872}]
I20260812 21:06:57.392475 6794 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53220:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 21:06:57.392578 6794 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53220:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 21:06:57.393260 6794 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 36a27cabe4ae44568b7adc99d86a1430: Sending DeleteTablet for 3 replicas of tablet d02b46993aa243c0a74732ae7ddd2db6
I20260812 21:06:57.393543 7428 tablet_service.cc:1514] Processing DeleteTablet for tablet d02b46993aa243c0a74732ae7ddd2db6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:06:57 UTC) from {username='kudu', principal='kudu/127.11.254.190@KRBTEST.COM'} at 127.0.0.1:35394
I20260812 21:06:57.393574 6990 tablet_service.cc:1514] Processing DeleteTablet for tablet d02b46993aa243c0a74732ae7ddd2db6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:06:57 UTC) from {username='kudu', principal='kudu/127.11.254.190@KRBTEST.COM'} at 127.0.0.1:51016
I20260812 21:06:57.393617 7204 tablet_service.cc:1514] Processing DeleteTablet for tablet d02b46993aa243c0a74732ae7ddd2db6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:06:57 UTC) from {username='kudu', principal='kudu/127.11.254.190@KRBTEST.COM'} at 127.0.0.1:49548
I20260812 21:06:57.393664 7103 tablet_replica.cc:330] T d02b46993aa243c0a74732ae7ddd2db6 P 813f1c245a4e49c499d584012bf01a52: stopping tablet replica
I20260812 21:06:57.393705 7103 raft_consensus.cc:2229] T d02b46993aa243c0a74732ae7ddd2db6 P 813f1c245a4e49c499d584012bf01a52 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 21:06:57.393743 7103 raft_consensus.cc:2258] T d02b46993aa243c0a74732ae7ddd2db6 P 813f1c245a4e49c499d584012bf01a52 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 21:06:57.393863 7588 tablet_replica.cc:330] T d02b46993aa243c0a74732ae7ddd2db6 P 024bc23e991c46bc813212a50b07f065: stopping tablet replica
I20260812 21:06:57.393886 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 6852
I20260812 21:06:57.405571 7590 tablet_replica.cc:330] T d02b46993aa243c0a74732ae7ddd2db6 P 8eb6c52ef06a4d5f97762c59705d0047: stopping tablet replica
I20260812 21:06:57.414462 7590 raft_consensus.cc:2229] T d02b46993aa243c0a74732ae7ddd2db6 P 8eb6c52ef06a4d5f97762c59705d0047 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 21:06:57.414744 7590 raft_consensus.cc:2258] T d02b46993aa243c0a74732ae7ddd2db6 P 8eb6c52ef06a4d5f97762c59705d0047 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 21:06:57.394001 7588 raft_consensus.cc:2229] T d02b46993aa243c0a74732ae7ddd2db6 P 024bc23e991c46bc813212a50b07f065 [term 1 LEADER]: Raft consensus shutting down.
I20260812 21:06:57.422958 7588 raft_consensus.cc:2258] T d02b46993aa243c0a74732ae7ddd2db6 P 024bc23e991c46bc813212a50b07f065 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 21:06:57.423722 7590 ts_tablet_manager.cc:1905] T d02b46993aa243c0a74732ae7ddd2db6 P 8eb6c52ef06a4d5f97762c59705d0047: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 21:06:57.425655 7590 ts_tablet_manager.cc:1918] T d02b46993aa243c0a74732ae7ddd2db6 P 8eb6c52ef06a4d5f97762c59705d0047: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 21:06:57.425742 7590 log.cc:1196] T d02b46993aa243c0a74732ae7ddd2db6 P 8eb6c52ef06a4d5f97762c59705d0047: Deleting WAL directory at /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/ts-1/wal/wals/d02b46993aa243c0a74732ae7ddd2db6
I20260812 21:06:57.425900 7588 ts_tablet_manager.cc:1905] T d02b46993aa243c0a74732ae7ddd2db6 P 024bc23e991c46bc813212a50b07f065: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 21:06:57.426120 7590 ts_tablet_manager.cc:1939] T d02b46993aa243c0a74732ae7ddd2db6 P 8eb6c52ef06a4d5f97762c59705d0047: Deleting consensus metadata
I20260812 21:06:57.426673 6771 catalog_manager.cc:4841] TS 8eb6c52ef06a4d5f97762c59705d0047 (127.11.254.130:39975): tablet d02b46993aa243c0a74732ae7ddd2db6 (table test-table [id=9f8260ba21404588b1419dfa4639b806]) successfully deleted
I20260812 21:06:57.427623 7588 ts_tablet_manager.cc:1918] T d02b46993aa243c0a74732ae7ddd2db6 P 024bc23e991c46bc813212a50b07f065: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 21:06:57.427700 7588 log.cc:1196] T d02b46993aa243c0a74732ae7ddd2db6 P 024bc23e991c46bc813212a50b07f065: Deleting WAL directory at /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/ts-2/wal/wals/d02b46993aa243c0a74732ae7ddd2db6
W20260812 21:06:57.427953 6766 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.254.129:38685 (error 108)
I20260812 21:06:57.427997 7588 ts_tablet_manager.cc:1939] T d02b46993aa243c0a74732ae7ddd2db6 P 024bc23e991c46bc813212a50b07f065: Deleting consensus metadata
I20260812 21:06:57.428443 6766 catalog_manager.cc:4841] TS 024bc23e991c46bc813212a50b07f065 (127.11.254.131:38309): tablet d02b46993aa243c0a74732ae7ddd2db6 (table test-table [id=9f8260ba21404588b1419dfa4639b806]) successfully deleted
W20260812 21:06:57.428773 7551 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:06:57.428399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.129:38685 (local address 127.0.0.1:51026)
0812 21:06:57.428453 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0812 21:06:57.428465 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 21:06:57.428508 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:06:57.428730 (+ 222us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.129:38685: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":31}
I20260812 21:06:57.428974 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 7096
W20260812 21:06:57.428918 6766 catalog_manager.cc:4584] TS 813f1c245a4e49c499d584012bf01a52 (127.11.254.129:38685): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d02b46993aa243c0a74732ae7ddd2db6: Network error: Client connection negotiation failed: client connection to 127.11.254.129:38685: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260812 21:06:57.439057 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 7317
I20260812 21:06:57.451761 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 6740
I20260812 21:07:17.470656 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.254.190:38123
--webserver_interface=127.11.254.190
--webserver_port=33283
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.190.keytab
--principal=kudu/127.11.254.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.254.148:34711
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.254.190:38123
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:07:17.554730 7593 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:07:17.554955 7593 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:07:17.554977 7593 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:07:17.556145 7593 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260812 21:07:17.556185 7593 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260812 21:07:17.556195 7593 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260812 21:07:17.556206 7593 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 21:07:17.556218 7593 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:07:17.556231 7593 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:07:17.556241 7593 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 21:07:17.556254 7593 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 21:07:17.557781 7593 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.254.148:34711
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.11.254.190:38123
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.190.keytab
--principal=kudu/127.11.254.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.254.190:38123
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.254.190
--webserver_port=33283
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7593
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:07:17.558044 7593 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:07:17.558248 7593 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:07:17.563529 7607 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:07:17.563537 7603 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:07:17.563583 7593 server_base.cc:1034] running on GCE node
W20260812 21:07:17.563586 7604 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:07:17.563979 7593 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:07:17.564297 7593 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:07:17.565445 7593 hybrid_clock.cc:648] HybridClock initialized: now 1786568837565426 us; error 39 us; skew 500 ppm
Aug 12 21:07:17 dist-test-slave-vpvm krb5kdc[5864](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568837, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:07:17.567976 7593 init.cc:377] Logged in from keytab as kudu/127.11.254.190@KRBTEST.COM (short username kudu)
I20260812 21:07:17.568442 7593 webserver.cc:460] Webserver started at http://127.11.254.190:33283/ using document root <none> and password file <none>
I20260812 21:07:17.568670 7593 fs_manager.cc:359] Metadata directory not provided
I20260812 21:07:17.568715 7593 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:07:17.570148 7593 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 21:07:17.570933 7621 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:07:17.571087 7593 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 21:07:17.571172 7593 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/master-0/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/master-0/wal
uuid: "36a27cabe4ae44568b7adc99d86a1430"
format_stamp: "Formatted at 2026-08-12 21:06:35 on dist-test-slave-vpvm"
server_key: "0c538b3805728f70b6f07f44ee6460b6"
server_key_iv: "5f1b651e34f771fe03d7352717eea675"
server_key_version: "encryptionkey@0"
I20260812 21:07:17.571585 7593 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:07:17.589564 7593 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:07:17.591717 7593 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:07:17.591851 7593 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:07:17.596103 7593 rpc_server.cc:307] RPC server started. Bound to: 127.11.254.190:38123
I20260812 21:07:17.596139 7698 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.254.190:38123 every 8 connection(s)
I20260812 21:07:17.596531 7593 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
I20260812 21:07:17.598098 7700 sys_catalog.cc:263] Verifying existing consensus state
I20260812 21:07:17.598783 7700 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36a27cabe4ae44568b7adc99d86a1430: Bootstrap starting.
I20260812 21:07:17.606877 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 7593
I20260812 21:07:17.607290 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.254.129:38685
--local_ip_for_outbound_sockets=127.11.254.129
--tserver_master_addrs=127.11.254.190:38123
--webserver_port=34443
--webserver_interface=127.11.254.129
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.129.keytab
--principal=kudu/127.11.254.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.254.148:34711
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 21:07:17.608618 7700 log.cc:824] T 00000000000000000000000000000000 P 36a27cabe4ae44568b7adc99d86a1430: Log is configured to *not* fsync() on all Append() calls
I20260812 21:07:17.611486 7700 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36a27cabe4ae44568b7adc99d86a1430: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 21:07:17.611845 7700 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36a27cabe4ae44568b7adc99d86a1430: Bootstrap complete.
I20260812 21:07:17.615348 7700 raft_consensus.cc:348] T 00000000000000000000000000000000 P 36a27cabe4ae44568b7adc99d86a1430 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36a27cabe4ae44568b7adc99d86a1430" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 38123 } }
I20260812 21:07:17.615701 7700 raft_consensus.cc:729] T 00000000000000000000000000000000 P 36a27cabe4ae44568b7adc99d86a1430 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36a27cabe4ae44568b7adc99d86a1430, State: Initialized, Role: FOLLOWER
I20260812 21:07:17.615885 7700 consensus_queue.cc:260] T 00000000000000000000000000000000 P 36a27cabe4ae44568b7adc99d86a1430 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36a27cabe4ae44568b7adc99d86a1430" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 38123 } }
I20260812 21:07:17.615967 7700 raft_consensus.cc:388] T 00000000000000000000000000000000 P 36a27cabe4ae44568b7adc99d86a1430 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 21:07:17.616019 7700 raft_consensus.cc:482] T 00000000000000000000000000000000 P 36a27cabe4ae44568b7adc99d86a1430 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 21:07:17.616080 7700 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 36a27cabe4ae44568b7adc99d86a1430 [term 2 FOLLOWER]: Advancing to term 3
I20260812 21:07:17.617583 7700 raft_consensus.cc:504] T 00000000000000000000000000000000 P 36a27cabe4ae44568b7adc99d86a1430 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36a27cabe4ae44568b7adc99d86a1430" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 38123 } }
I20260812 21:07:17.617712 7700 leader_election.cc:302] T 00000000000000000000000000000000 P 36a27cabe4ae44568b7adc99d86a1430 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 36a27cabe4ae44568b7adc99d86a1430; no voters:
I20260812 21:07:17.617920 7700 leader_election.cc:288] T 00000000000000000000000000000000 P 36a27cabe4ae44568b7adc99d86a1430 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260812 21:07:17.618028 7704 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 36a27cabe4ae44568b7adc99d86a1430 [term 3 FOLLOWER]: Leader election won for term 3
I20260812 21:07:17.618251 7700 sys_catalog.cc:564] T 00000000000000000000000000000000 P 36a27cabe4ae44568b7adc99d86a1430 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 21:07:17.618275 7704 raft_consensus.cc:686] T 00000000000000000000000000000000 P 36a27cabe4ae44568b7adc99d86a1430 [term 3 LEADER]: Becoming Leader. State: Replica: 36a27cabe4ae44568b7adc99d86a1430, State: Running, Role: LEADER
I20260812 21:07:17.618384 7704 consensus_queue.cc:237] T 00000000000000000000000000000000 P 36a27cabe4ae44568b7adc99d86a1430 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36a27cabe4ae44568b7adc99d86a1430" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 38123 } }
I20260812 21:07:17.618722 7704 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36a27cabe4ae44568b7adc99d86a1430 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 36a27cabe4ae44568b7adc99d86a1430. Latest consensus state: current_term: 3 leader_uuid: "36a27cabe4ae44568b7adc99d86a1430" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36a27cabe4ae44568b7adc99d86a1430" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 38123 } } }
I20260812 21:07:17.618809 7704 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36a27cabe4ae44568b7adc99d86a1430 [sys.catalog]: This master's current role is: LEADER
I20260812 21:07:17.619098 7705 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36a27cabe4ae44568b7adc99d86a1430 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "36a27cabe4ae44568b7adc99d86a1430" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36a27cabe4ae44568b7adc99d86a1430" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 38123 } } }
I20260812 21:07:17.619184 7705 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36a27cabe4ae44568b7adc99d86a1430 [sys.catalog]: This master's current role is: LEADER
I20260812 21:07:17.619310 7720 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 21:07:17.620255 7720 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 21:07:17.620447 7720 catalog_manager.cc:1259] Loaded cluster ID: 27a149e698fb48b0a6d33906f55215fa
I20260812 21:07:17.620662 7720 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 21:07:17.621843 7720 catalog_manager.cc:1504] Loading token signing keys...
I20260812 21:07:17.622143 7720 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 36a27cabe4ae44568b7adc99d86a1430: Loaded TSK: 0
I20260812 21:07:17.622207 7720 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 36a27cabe4ae44568b7adc99d86a1430: Loaded TSK: 1
F20260812 21:07:17.622476 7720 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7faaf168a9ad google::LogMessage::Fail() at ??:0
@ 0x7faaf168d88a google::LogMessage::SendToLog() at ??:0
@ 0x7faaf168a4de google::LogMessage::Flush() at ??:0
@ 0x7faaf168ba19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7faaf3df7104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7faaf3df4d41 kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7faaf95e41cd kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7faaf95e448e kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7faaf95e4594 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7faaf95aac48 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7faaf95b4ad0 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7faaf35580d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7faaf354d954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7faaf43b16db start_thread at ??:0
@ 0x7faaf018861f clone at ??:0
W20260812 21:07:17.716679 7703 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:07:17.716894 7703 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:07:17.716919 7703 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:07:17.718151 7703 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:07:17.718194 7703 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:07:17.718221 7703 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.254.129
I20260812 21:07:17.719756 7703 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.254.148:34711
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.129.keytab
--principal=kudu/127.11.254.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.254.129:38685
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.254.129
--webserver_port=34443
--tserver_master_addrs=127.11.254.190:38123
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7703
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.254.129
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:07:17.720000 7703 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:07:17.720239 7703 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:07:17.726647 7770 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:07:17.726605 7767 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:07:17.726614 7766 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:07:17.726897 7703 server_base.cc:1034] running on GCE node
I20260812 21:07:17.727097 7703 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:07:17.727339 7703 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:07:17.728499 7703 hybrid_clock.cc:648] HybridClock initialized: now 1786568837728483 us; error 34 us; skew 500 ppm
Aug 12 21:07:17 dist-test-slave-vpvm krb5kdc[5864](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568837, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:07:17.731079 7703 init.cc:377] Logged in from keytab as kudu/127.11.254.129@KRBTEST.COM (short username kudu)
I20260812 21:07:17.731551 7703 webserver.cc:460] Webserver started at http://127.11.254.129:34443/ using document root <none> and password file <none>
I20260812 21:07:17.731755 7703 fs_manager.cc:359] Metadata directory not provided
I20260812 21:07:17.731799 7703 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:07:17.733206 7703 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 21:07:17.733982 7779 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:07:17.734170 7703 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 21:07:17.734254 7703 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/ts-0/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/ts-0/wal
uuid: "813f1c245a4e49c499d584012bf01a52"
format_stamp: "Formatted at 2026-08-12 21:06:36 on dist-test-slave-vpvm"
server_key: "a606426d65f79390689a46a5466b23a5"
server_key_iv: "efa852fc71081e51e3bd8973ee227f20"
server_key_version: "encryptionkey@0"
I20260812 21:07:17.734726 7703 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:07:17.766248 7703 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:07:17.769106 7703 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:07:17.769287 7703 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:07:17.769604 7703 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:07:17.770252 7791 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260812 21:07:17.772300 7703 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 21:07:17.772367 7703 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260812 21:07:17.772406 7703 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260812 21:07:17.773131 7703 ts_tablet_manager.cc:610] Registered 1 tablets
I20260812 21:07:17.773167 7703 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260812 21:07:17.773272 7791 tablet_bootstrap.cc:492] T d02b46993aa243c0a74732ae7ddd2db6 P 813f1c245a4e49c499d584012bf01a52: Bootstrap starting.
I20260812 21:07:17.778856 7703 rpc_server.cc:307] RPC server started. Bound to: 127.11.254.129:38685
I20260812 21:07:17.779435 7703 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/ts-0/data/info.pb
I20260812 21:07:17.780298 7964 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.254.129:38685 every 8 connection(s)
I20260812 21:07:17.784859 7791 log.cc:824] T d02b46993aa243c0a74732ae7ddd2db6 P 813f1c245a4e49c499d584012bf01a52: Log is configured to *not* fsync() on all Append() calls
I20260812 21:07:17.785048 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 7703
I20260812 21:07:17.785390 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.254.130:39975
--local_ip_for_outbound_sockets=127.11.254.130
--tserver_master_addrs=127.11.254.190:38123
--webserver_port=43347
--webserver_interface=127.11.254.130
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.130.keytab
--principal=kudu/127.11.254.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.254.148:34711
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 21:07:17.787923 7791 tablet_bootstrap.cc:492] T d02b46993aa243c0a74732ae7ddd2db6 P 813f1c245a4e49c499d584012bf01a52: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 21:07:17.788226 7791 tablet_bootstrap.cc:492] T d02b46993aa243c0a74732ae7ddd2db6 P 813f1c245a4e49c499d584012bf01a52: Bootstrap complete.
I20260812 21:07:17.789053 7791 ts_tablet_manager.cc:1397] T d02b46993aa243c0a74732ae7ddd2db6 P 813f1c245a4e49c499d584012bf01a52: Time spent bootstrapping tablet: real 0.016s user 0.003s sys 0.009s
I20260812 21:07:17.790205 7791 raft_consensus.cc:348] T d02b46993aa243c0a74732ae7ddd2db6 P 813f1c245a4e49c499d584012bf01a52 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "024bc23e991c46bc813212a50b07f065" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 38309 } } peers { permanent_uuid: "8eb6c52ef06a4d5f97762c59705d0047" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 39975 } } peers { permanent_uuid: "813f1c245a4e49c499d584012bf01a52" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 38685 } }
I20260812 21:07:17.790575 7791 raft_consensus.cc:729] T d02b46993aa243c0a74732ae7ddd2db6 P 813f1c245a4e49c499d584012bf01a52 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 813f1c245a4e49c499d584012bf01a52, State: Initialized, Role: FOLLOWER
I20260812 21:07:17.790753 7791 consensus_queue.cc:260] T d02b46993aa243c0a74732ae7ddd2db6 P 813f1c245a4e49c499d584012bf01a52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "024bc23e991c46bc813212a50b07f065" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 38309 } } peers { permanent_uuid: "8eb6c52ef06a4d5f97762c59705d0047" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 39975 } } peers { permanent_uuid: "813f1c245a4e49c499d584012bf01a52" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 38685 } }
I20260812 21:07:17.791087 7791 ts_tablet_manager.cc:1428] T d02b46993aa243c0a74732ae7ddd2db6 P 813f1c245a4e49c499d584012bf01a52: Time spent starting tablet: real 0.002s user 0.005s sys 0.000s
W20260812 21:07:17.882195 7968 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:07:17.780553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:38123 (local address 127.11.254.129:51691)
0812 21:07:17.780741 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0812 21:07:17.780787 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 21:07:17.781272 (+ 485us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:07:17.881852 (+100580us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.190:38123: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":84,"threads_started":1}
W20260812 21:07:17.882476 7965 heartbeater.cc:644] Failed to heartbeat to 127.11.254.190:38123 (0 consecutive failures): Network error: Failed to ping master at 127.11.254.190:38123: Client connection negotiation failed: client connection to 127.11.254.190:38123: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260812 21:07:17.882813 7968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:17.882699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:38123 (local address 127.11.254.129:58605)
0812 21:07:17.882751 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0812 21:07:17.882792 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.190:38123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260812 21:07:17.883245 7968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:17.883121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:38123 (local address 127.11.254.129:38561)
0812 21:07:17.883195 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0812 21:07:17.883225 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.190:38123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260812 21:07:17.883376 7965 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 21:07:17.892820 7970 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:07:17.893039 7970 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:07:17.893074 7970 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:07:17.894313 7970 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:07:17.894359 7970 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:07:17.894395 7970 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.254.130
I20260812 21:07:17.895941 7970 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.254.148:34711
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.130.keytab
--principal=kudu/127.11.254.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.254.130:39975
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.254.130
--webserver_port=43347
--tserver_master_addrs=127.11.254.190:38123
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7970
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.254.130
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:07:17.896188 7970 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:07:17.896414 7970 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:07:17.901640 7981 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:07:17.901794 7970 server_base.cc:1034] running on GCE node
W20260812 21:07:17.901647 7984 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:07:17.901647 7988 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:07:17.902336 7970 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:07:17.902597 7970 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:07:17.903779 7970 hybrid_clock.cc:648] HybridClock initialized: now 1786568837903761 us; error 29 us; skew 500 ppm
Aug 12 21:07:17 dist-test-slave-vpvm krb5kdc[5864](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568837, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:07:17.906796 7970 init.cc:377] Logged in from keytab as kudu/127.11.254.130@KRBTEST.COM (short username kudu)
I20260812 21:07:17.907336 7970 webserver.cc:460] Webserver started at http://127.11.254.130:43347/ using document root <none> and password file <none>
I20260812 21:07:17.907588 7970 fs_manager.cc:359] Metadata directory not provided
I20260812 21:07:17.907644 7970 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:07:17.909103 7970 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 21:07:17.909865 7999 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:07:17.910044 7970 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 21:07:17.910156 7970 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/ts-1/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/ts-1/wal
uuid: "8eb6c52ef06a4d5f97762c59705d0047"
format_stamp: "Formatted at 2026-08-12 21:06:36 on dist-test-slave-vpvm"
server_key: "ec49e5db1cd29defa3404affc9b1cc98"
server_key_iv: "bc96b50eff0bf064cfb099c763c04d06"
server_key_version: "encryptionkey@0"
I20260812 21:07:17.910555 7970 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:07:17.922703 7970 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:07:17.924814 7970 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:07:17.924973 7970 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:07:17.925215 7970 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:07:17.925578 7970 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:07:17.925616 7970 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:07:17.925657 7970 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:07:17.925680 7970 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:07:17.931710 7970 rpc_server.cc:307] RPC server started. Bound to: 127.11.254.130:39975
I20260812 21:07:17.931772 8181 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.254.130:39975 every 8 connection(s)
I20260812 21:07:17.932169 7970 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/ts-1/data/info.pb
I20260812 21:07:17.933318 8186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:17.932898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:38123 (local address 127.11.254.130:55969)
0812 21:07:17.933091 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0812 21:07:17.933172 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.190:38123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":93,"threads_started":1}
W20260812 21:07:17.933733 8182 heartbeater.cc:644] Failed to heartbeat to 127.11.254.190:38123 (0 consecutive failures): Network error: Failed to ping master at 127.11.254.190:38123: Client connection negotiation failed: client connection to 127.11.254.190:38123: connect: Connection refused (error 111)
I20260812 21:07:17.934033 8186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:17.933919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:38123 (local address 127.11.254.130:33753)
0812 21:07:17.933983 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0812 21:07:17.934010 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.190:38123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260812 21:07:17.934327 8186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:17.934222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:38123 (local address 127.11.254.130:58937)
0812 21:07:17.934286 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0812 21:07:17.934310 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.190:38123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260812 21:07:17.934425 8182 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260812 21:07:17.939348 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 7970
I20260812 21:07:17.939634 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.254.131:38309
--local_ip_for_outbound_sockets=127.11.254.131
--tserver_master_addrs=127.11.254.190:38123
--webserver_port=46091
--webserver_interface=127.11.254.131
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.131.keytab
--principal=kudu/127.11.254.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.254.148:34711
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:07:18.022995 8187 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:07:18.023258 8187 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:07:18.023285 8187 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:07:18.024474 8187 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:07:18.024518 8187 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:07:18.024547 8187 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.254.131
I20260812 21:07:18.026084 8187 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.254.148:34711
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.131.keytab
--principal=kudu/127.11.254.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.254.131:38309
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.254.131
--webserver_port=46091
--tserver_master_addrs=127.11.254.190:38123
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8187
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.254.131
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:07:18.026341 8187 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:07:18.026587 8187 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 21:07:18.032138 8187 server_base.cc:1034] running on GCE node
W20260812 21:07:18.032066 8194 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:07:18.032066 8195 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:07:18.032094 8199 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:07:18.032528 8187 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:07:18.032796 8187 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:07:18.033962 8187 hybrid_clock.cc:648] HybridClock initialized: now 1786568838033931 us; error 27 us; skew 500 ppm
Aug 12 21:07:18 dist-test-slave-vpvm krb5kdc[5864](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568838, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:07:18.036681 8187 init.cc:377] Logged in from keytab as kudu/127.11.254.131@KRBTEST.COM (short username kudu)
I20260812 21:07:18.037151 8187 webserver.cc:460] Webserver started at http://127.11.254.131:46091/ using document root <none> and password file <none>
I20260812 21:07:18.037364 8187 fs_manager.cc:359] Metadata directory not provided
I20260812 21:07:18.037415 8187 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:07:18.038937 8187 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 21:07:18.039557 8210 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:07:18.039723 8187 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 21:07:18.039820 8187 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/ts-2/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/ts-2/wal
uuid: "024bc23e991c46bc813212a50b07f065"
format_stamp: "Formatted at 2026-08-12 21:06:36 on dist-test-slave-vpvm"
server_key: "7a0d4318ad00daca63381859cea667cd"
server_key_iv: "319262ae1c58b20aebe38eb1be4a839d"
server_key_version: "encryptionkey@0"
I20260812 21:07:18.040195 8187 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:07:18.061540 8187 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:07:18.063843 8187 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:07:18.063995 8187 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:07:18.064236 8187 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:07:18.064543 8187 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:07:18.064584 8187 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:07:18.064615 8187 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:07:18.064632 8187 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:07:18.070336 8187 rpc_server.cc:307] RPC server started. Bound to: 127.11.254.131:38309
I20260812 21:07:18.070398 8399 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.254.131:38309 every 8 connection(s)
I20260812 21:07:18.070822 8187 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786568607354921-12282-0/minicluster-data/ts-2/data/info.pb
I20260812 21:07:18.071837 8407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:18.071451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:38123 (local address 127.11.254.131:38231)
0812 21:07:18.071599 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0812 21:07:18.071689 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.190:38123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":70,"threads_started":1}
W20260812 21:07:18.072235 8402 heartbeater.cc:644] Failed to heartbeat to 127.11.254.190:38123 (0 consecutive failures): Network error: Failed to ping master at 127.11.254.190:38123: Client connection negotiation failed: client connection to 127.11.254.190:38123: connect: Connection refused (error 111)
I20260812 21:07:18.072475 8407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:18.072391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:38123 (local address 127.11.254.131:57769)
0812 21:07:18.072430 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0812 21:07:18.072458 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.190:38123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260812 21:07:18.072736 8407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:18.072667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:38123 (local address 127.11.254.131:47829)
0812 21:07:18.072700 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0812 21:07:18.072721 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.190:38123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
W20260812 21:07:18.072825 8402 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260812 21:07:18.075654 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 8187
I20260812 21:07:18.884074 8411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:18.883790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:38123 (local address 127.11.254.129:41433)
0812 21:07:18.883973 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0812 21:07:18.884018 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.190:38123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":68,"threads_started":1}
I20260812 21:07:18.935050 8412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:18.934786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:38123 (local address 127.11.254.130:39543)
0812 21:07:18.934931 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0812 21:07:18.934972 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.190:38123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":65,"threads_started":1}
I20260812 21:07:19.073393 8413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:19.073175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:38123 (local address 127.11.254.131:51399)
0812 21:07:19.073302 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0812 21:07:19.073338 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.190:38123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":62,"threads_started":1}
I20260812 21:07:19.528033 8415 raft_consensus.cc:482] T d02b46993aa243c0a74732ae7ddd2db6 P 813f1c245a4e49c499d584012bf01a52 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 21:07:19.528189 8415 raft_consensus.cc:504] T d02b46993aa243c0a74732ae7ddd2db6 P 813f1c245a4e49c499d584012bf01a52 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "024bc23e991c46bc813212a50b07f065" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 38309 } } peers { permanent_uuid: "8eb6c52ef06a4d5f97762c59705d0047" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 39975 } } peers { permanent_uuid: "813f1c245a4e49c499d584012bf01a52" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 38685 } }
I20260812 21:07:19.528563 8415 leader_election.cc:288] T d02b46993aa243c0a74732ae7ddd2db6 P 813f1c245a4e49c499d584012bf01a52 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 024bc23e991c46bc813212a50b07f065 (127.11.254.131:38309), 8eb6c52ef06a4d5f97762c59705d0047 (127.11.254.130:39975)
Aug 12 21:07:19 dist-test-slave-vpvm krb5kdc[5864](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568837, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.129@KRBTEST.COM for kudu/127.11.254.130@KRBTEST.COM
Aug 12 21:07:19 dist-test-slave-vpvm krb5kdc[5864](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568837, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.129@KRBTEST.COM for kudu/127.11.254.131@KRBTEST.COM
I20260812 21:07:19.541280 8420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:19.528762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.130:39975 (local address 127.11.254.129:60819)
0812 21:07:19.528907 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0812 21:07:19.528919 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 21:07:19.528978 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:07:19.530015 (+ 1037us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:07:19.530015 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:07:19.530462 (+ 447us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:07:19.530743 (+ 281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:07:19.530751 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:07:19.532460 (+ 1709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:07:19.532460 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:07:19.533767 (+ 1307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:07:19.533770 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:07:19.534660 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:07:19.534661 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:07:19.534825 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:07:19.535420 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:07:19.535431 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:07:19.537311 (+ 1880us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:07:19.539422 (+ 2111us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:07:19.539430 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:07:19.539432 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:07:19.539761 (+ 329us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:07:19.540071 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:07:19.540072 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:07:19.540073 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:07:19.540198 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:07:19.540676 (+ 478us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:07:19.540683 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:07:19.540852 (+ 169us) client_negotiation.cc:769] Sending connection context
0812 21:07:19.540971 (+ 119us) client_negotiation.cc:240] Negotiation successful
0812 21:07:19.541161 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"mutex_wait_us":112,"thread_start_us":52,"threads_started":1}
I20260812 21:07:19.541296 8423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:19.528867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.129:46975 (local address 127.11.254.131:38309)
0812 21:07:19.529079 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0812 21:07:19.529086 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 21:07:19.529117 (+ 31us) server_negotiation.cc:407] Connection header received
0812 21:07:19.529679 (+ 562us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:07:19.529682 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:07:19.529738 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:07:19.529890 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:07:19.530872 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:19.532302 (+ 1430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:19.533877 (+ 1575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:19.534562 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:19.534722 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:07:19.537738 (+ 3016us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:07:19.537743 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:07:19.537743 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:07:19.537756 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:07:19.539523 (+ 1767us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:07:19.539873 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:07:19.539876 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:07:19.539876 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:07:19.539985 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:07:19.540260 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:07:19.540261 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:07:19.540261 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:07:19.540602 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:07:19.540692 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0812 21:07:19.541034 (+ 342us) server_negotiation.cc:299] Negotiation successful
0812 21:07:19.541178 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":73,"threads_started":1}
I20260812 21:07:19.541296 8421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:19.528872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.129:60819 (local address 127.11.254.130:39975)
0812 21:07:19.529021 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0812 21:07:19.529034 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 21:07:19.529060 (+ 26us) server_negotiation.cc:407] Connection header received
0812 21:07:19.529668 (+ 608us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:07:19.529672 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:07:19.529734 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:07:19.529890 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:07:19.530872 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:19.532302 (+ 1430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:19.533867 (+ 1565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:19.534569 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:19.534700 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:07:19.537421 (+ 2721us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:07:19.537426 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:07:19.537427 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:07:19.537441 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:07:19.539340 (+ 1899us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:07:19.539857 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:07:19.539861 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:07:19.539861 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:07:19.539985 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:07:19.540260 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:07:19.540260 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:07:19.540261 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:07:19.540602 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:07:19.540692 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0812 21:07:19.541034 (+ 342us) server_negotiation.cc:299] Negotiation successful
0812 21:07:19.541178 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":49,"threads_started":1}
I20260812 21:07:19.541334 8417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:19.528715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.131:38309 (local address 127.11.254.129:46975)
0812 21:07:19.528877 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0812 21:07:19.528888 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 21:07:19.528949 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:07:19.529986 (+ 1037us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:07:19.529989 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:07:19.530473 (+ 484us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:07:19.530743 (+ 270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:07:19.530751 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:07:19.532445 (+ 1694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:07:19.532446 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:07:19.533767 (+ 1321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:07:19.533770 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:07:19.534695 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:07:19.534696 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:07:19.534824 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:07:19.535434 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:07:19.535438 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:07:19.537625 (+ 2187us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:07:19.539596 (+ 1971us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:07:19.539597 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:07:19.539597 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:07:19.539804 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:07:19.540046 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:07:19.540047 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:07:19.540047 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:07:19.540198 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:07:19.540676 (+ 478us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:07:19.540683 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:07:19.540852 (+ 169us) client_negotiation.cc:769] Sending connection context
0812 21:07:19.540971 (+ 119us) client_negotiation.cc:240] Negotiation successful
0812 21:07:19.541227 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"mutex_wait_us":166,"thread_start_us":107,"threads_started":1}
W20260812 21:07:19.542433 7782 leader_election.cc:341] T d02b46993aa243c0a74732ae7ddd2db6 P 813f1c245a4e49c499d584012bf01a52 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 024bc23e991c46bc813212a50b07f065 (127.11.254.131:38309): Not found: Tablet not found: d02b46993aa243c0a74732ae7ddd2db6
W20260812 21:07:19.542519 7788 leader_election.cc:341] T d02b46993aa243c0a74732ae7ddd2db6 P 813f1c245a4e49c499d584012bf01a52 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8eb6c52ef06a4d5f97762c59705d0047 (127.11.254.130:39975): Not found: Tablet not found: d02b46993aa243c0a74732ae7ddd2db6
I20260812 21:07:19.542605 7788 leader_election.cc:302] T d02b46993aa243c0a74732ae7ddd2db6 P 813f1c245a4e49c499d584012bf01a52 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 813f1c245a4e49c499d584012bf01a52; no voters: 024bc23e991c46bc813212a50b07f065, 8eb6c52ef06a4d5f97762c59705d0047
I20260812 21:07:19.542712 8415 raft_consensus.cc:2726] T d02b46993aa243c0a74732ae7ddd2db6 P 813f1c245a4e49c499d584012bf01a52 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 21:07:19.884912 8417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:19.884731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:38123 (local address 127.11.254.129:35535)
0812 21:07:19.884811 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0812 21:07:19.884864 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.190:38123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260812 21:07:19.935948 8424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:19.935665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:38123 (local address 127.11.254.130:55489)
0812 21:07:19.935835 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0812 21:07:19.935895 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.190:38123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":86,"threads_started":1}
I20260812 21:07:20.074241 8425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:20.073931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:38123 (local address 127.11.254.131:33033)
0812 21:07:20.074148 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0812 21:07:20.074194 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.190:38123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":130,"threads_started":1}
I20260812 21:07:20.885769 8427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:20.885491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:38123 (local address 127.11.254.129:44775)
0812 21:07:20.885673 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0812 21:07:20.885717 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.190:38123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":77,"threads_started":1}
I20260812 21:07:20.936611 8428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:20.936366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:38123 (local address 127.11.254.130:47399)
0812 21:07:20.936490 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0812 21:07:20.936527 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.190:38123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":67,"threads_started":1}
I20260812 21:07:21.074973 8429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:21.074731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:38123 (local address 127.11.254.131:60403)
0812 21:07:21.074879 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0812 21:07:21.074923 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.190:38123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":70,"threads_started":1}
I20260812 21:07:21.160943 8435 raft_consensus.cc:482] T d02b46993aa243c0a74732ae7ddd2db6 P 813f1c245a4e49c499d584012bf01a52 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 21:07:21.161055 8435 raft_consensus.cc:504] T d02b46993aa243c0a74732ae7ddd2db6 P 813f1c245a4e49c499d584012bf01a52 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "024bc23e991c46bc813212a50b07f065" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 38309 } } peers { permanent_uuid: "8eb6c52ef06a4d5f97762c59705d0047" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 39975 } } peers { permanent_uuid: "813f1c245a4e49c499d584012bf01a52" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 38685 } }
I20260812 21:07:21.161211 8435 leader_election.cc:288] T d02b46993aa243c0a74732ae7ddd2db6 P 813f1c245a4e49c499d584012bf01a52 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 024bc23e991c46bc813212a50b07f065 (127.11.254.131:38309), 8eb6c52ef06a4d5f97762c59705d0047 (127.11.254.130:39975)
W20260812 21:07:21.161623 7782 leader_election.cc:341] T d02b46993aa243c0a74732ae7ddd2db6 P 813f1c245a4e49c499d584012bf01a52 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 024bc23e991c46bc813212a50b07f065 (127.11.254.131:38309): Not found: Tablet not found: d02b46993aa243c0a74732ae7ddd2db6
W20260812 21:07:21.161724 7788 leader_election.cc:341] T d02b46993aa243c0a74732ae7ddd2db6 P 813f1c245a4e49c499d584012bf01a52 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8eb6c52ef06a4d5f97762c59705d0047 (127.11.254.130:39975): Not found: Tablet not found: d02b46993aa243c0a74732ae7ddd2db6
I20260812 21:07:21.161805 7788 leader_election.cc:302] T d02b46993aa243c0a74732ae7ddd2db6 P 813f1c245a4e49c499d584012bf01a52 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 813f1c245a4e49c499d584012bf01a52; no voters: 024bc23e991c46bc813212a50b07f065, 8eb6c52ef06a4d5f97762c59705d0047
I20260812 21:07:21.161893 8435 raft_consensus.cc:2726] T d02b46993aa243c0a74732ae7ddd2db6 P 813f1c245a4e49c499d584012bf01a52 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 21:07:21.886632 8436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:21.886335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:38123 (local address 127.11.254.129:45609)
0812 21:07:21.886526 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0812 21:07:21.886573 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.190:38123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":101,"threads_started":1}
I20260812 21:07:21.937449 8437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:21.937078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:38123 (local address 127.11.254.130:52847)
0812 21:07:21.937273 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0812 21:07:21.937325 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.190:38123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":84,"threads_started":1}
I20260812 21:07:22.075724 8438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:22.075488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:38123 (local address 127.11.254.131:39703)
0812 21:07:22.075644 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0812 21:07:22.075680 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.190:38123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":74,"threads_started":1}
I20260812 21:07:22.887475 8439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:22.887205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:38123 (local address 127.11.254.129:60983)
0812 21:07:22.887373 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0812 21:07:22.887423 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.190:38123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":66,"threads_started":1}
I20260812 21:07:22.938305 8440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:22.938072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:38123 (local address 127.11.254.130:59223)
0812 21:07:22.938207 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0812 21:07:22.938254 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.190:38123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":64,"threads_started":1}
I20260812 21:07:23.076521 8442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:23.076251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:38123 (local address 127.11.254.131:34007)
0812 21:07:23.076414 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0812 21:07:23.076460 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.190:38123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":67,"threads_started":1}
I20260812 21:07:23.086910 8443 raft_consensus.cc:482] T d02b46993aa243c0a74732ae7ddd2db6 P 813f1c245a4e49c499d584012bf01a52 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 21:07:23.087026 8443 raft_consensus.cc:504] T d02b46993aa243c0a74732ae7ddd2db6 P 813f1c245a4e49c499d584012bf01a52 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "024bc23e991c46bc813212a50b07f065" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 38309 } } peers { permanent_uuid: "8eb6c52ef06a4d5f97762c59705d0047" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 39975 } } peers { permanent_uuid: "813f1c245a4e49c499d584012bf01a52" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 38685 } }
I20260812 21:07:23.087205 8443 leader_election.cc:288] T d02b46993aa243c0a74732ae7ddd2db6 P 813f1c245a4e49c499d584012bf01a52 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 024bc23e991c46bc813212a50b07f065 (127.11.254.131:38309), 8eb6c52ef06a4d5f97762c59705d0047 (127.11.254.130:39975)
W20260812 21:07:23.087627 7788 leader_election.cc:341] T d02b46993aa243c0a74732ae7ddd2db6 P 813f1c245a4e49c499d584012bf01a52 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8eb6c52ef06a4d5f97762c59705d0047 (127.11.254.130:39975): Not found: Tablet not found: d02b46993aa243c0a74732ae7ddd2db6
W20260812 21:07:23.087718 7782 leader_election.cc:341] T d02b46993aa243c0a74732ae7ddd2db6 P 813f1c245a4e49c499d584012bf01a52 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 024bc23e991c46bc813212a50b07f065 (127.11.254.131:38309): Not found: Tablet not found: d02b46993aa243c0a74732ae7ddd2db6
I20260812 21:07:23.087764 7782 leader_election.cc:302] T d02b46993aa243c0a74732ae7ddd2db6 P 813f1c245a4e49c499d584012bf01a52 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 813f1c245a4e49c499d584012bf01a52; no voters: 024bc23e991c46bc813212a50b07f065, 8eb6c52ef06a4d5f97762c59705d0047
I20260812 21:07:23.087838 8443 raft_consensus.cc:2726] T d02b46993aa243c0a74732ae7ddd2db6 P 813f1c245a4e49c499d584012bf01a52 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 21:07:23.888156 8444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:23.887944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:38123 (local address 127.11.254.129:38411)
0812 21:07:23.888072 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0812 21:07:23.888109 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.190:38123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":66,"threads_started":1}
I20260812 21:07:23.939196 8448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:23.938883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:38123 (local address 127.11.254.130:49717)
0812 21:07:23.939094 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0812 21:07:23.939136 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.190:38123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":136,"threads_started":1}
I20260812 21:07:24.077392 8449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:24.077093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:38123 (local address 127.11.254.131:38243)
0812 21:07:24.077293 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0812 21:07:24.077339 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.190:38123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":100,"threads_started":1}
I20260812 21:07:24.888935 8451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:24.888666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:38123 (local address 127.11.254.129:44101)
0812 21:07:24.888834 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0812 21:07:24.888883 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.190:38123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":67,"threads_started":1}
I20260812 21:07:24.939924 8452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:24.939685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:38123 (local address 127.11.254.130:44671)
0812 21:07:24.939828 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0812 21:07:24.939872 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.190:38123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":79,"threads_started":1}
I20260812 21:07:25.078145 8453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:25.077890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:38123 (local address 127.11.254.131:33211)
0812 21:07:25.078035 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0812 21:07:25.078079 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.190:38123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":88,"threads_started":1}
I20260812 21:07:25.889679 8454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:25.889437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:38123 (local address 127.11.254.129:35333)
0812 21:07:25.889583 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0812 21:07:25.889628 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.190:38123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":66,"threads_started":1}
I20260812 21:07:25.940692 8456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:25.940428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:38123 (local address 127.11.254.130:60317)
0812 21:07:25.940596 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0812 21:07:25.940640 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.190:38123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":75,"threads_started":1}
I20260812 21:07:26.058158 8458 raft_consensus.cc:482] T d02b46993aa243c0a74732ae7ddd2db6 P 813f1c245a4e49c499d584012bf01a52 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 21:07:26.058264 8458 raft_consensus.cc:504] T d02b46993aa243c0a74732ae7ddd2db6 P 813f1c245a4e49c499d584012bf01a52 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "024bc23e991c46bc813212a50b07f065" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 38309 } } peers { permanent_uuid: "8eb6c52ef06a4d5f97762c59705d0047" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 39975 } } peers { permanent_uuid: "813f1c245a4e49c499d584012bf01a52" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 38685 } }
I20260812 21:07:26.058439 8458 leader_election.cc:288] T d02b46993aa243c0a74732ae7ddd2db6 P 813f1c245a4e49c499d584012bf01a52 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 024bc23e991c46bc813212a50b07f065 (127.11.254.131:38309), 8eb6c52ef06a4d5f97762c59705d0047 (127.11.254.130:39975)
W20260812 21:07:26.058863 7782 leader_election.cc:341] T d02b46993aa243c0a74732ae7ddd2db6 P 813f1c245a4e49c499d584012bf01a52 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 024bc23e991c46bc813212a50b07f065 (127.11.254.131:38309): Not found: Tablet not found: d02b46993aa243c0a74732ae7ddd2db6
W20260812 21:07:26.058944 7788 leader_election.cc:341] T d02b46993aa243c0a74732ae7ddd2db6 P 813f1c245a4e49c499d584012bf01a52 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8eb6c52ef06a4d5f97762c59705d0047 (127.11.254.130:39975): Not found: Tablet not found: d02b46993aa243c0a74732ae7ddd2db6
I20260812 21:07:26.058974 7788 leader_election.cc:302] T d02b46993aa243c0a74732ae7ddd2db6 P 813f1c245a4e49c499d584012bf01a52 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 813f1c245a4e49c499d584012bf01a52; no voters: 024bc23e991c46bc813212a50b07f065, 8eb6c52ef06a4d5f97762c59705d0047
I20260812 21:07:26.059043 8458 raft_consensus.cc:2726] T d02b46993aa243c0a74732ae7ddd2db6 P 813f1c245a4e49c499d584012bf01a52 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 21:07:26.078917 8460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:26.078638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:38123 (local address 127.11.254.131:42627)
0812 21:07:26.078810 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0812 21:07:26.078857 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.190:38123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":69,"threads_started":1}
I20260812 21:07:26.890446 8461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:26.890184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:38123 (local address 127.11.254.129:34367)
0812 21:07:26.890346 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0812 21:07:26.890394 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.190:38123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":75,"threads_started":1}
I20260812 21:07:26.941329 8463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:26.941117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:38123 (local address 127.11.254.130:58631)
0812 21:07:26.941240 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0812 21:07:26.941280 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.190:38123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":66,"threads_started":1}
I20260812 21:07:27.079689 8464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:27.079442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:38123 (local address 127.11.254.131:52059)
0812 21:07:27.079592 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0812 21:07:27.079638 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.190:38123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":74,"threads_started":1}
I20260812 21:07:27.891379 8466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:27.891058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:38123 (local address 127.11.254.129:34963)
0812 21:07:27.891281 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0812 21:07:27.891327 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.190:38123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":150,"threads_started":1}
I20260812 21:07:27.942202 8467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:27.941900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:38123 (local address 127.11.254.130:43151)
0812 21:07:27.942078 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0812 21:07:27.942135 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.190:38123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":110,"threads_started":1}
I20260812 21:07:28.080541 8470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:28.080224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:38123 (local address 127.11.254.131:39253)
0812 21:07:28.080436 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0812 21:07:28.080488 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.190:38123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":73,"threads_started":1}
I20260812 21:07:28.751820 8471 raft_consensus.cc:482] T d02b46993aa243c0a74732ae7ddd2db6 P 813f1c245a4e49c499d584012bf01a52 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 21:07:28.751932 8471 raft_consensus.cc:504] T d02b46993aa243c0a74732ae7ddd2db6 P 813f1c245a4e49c499d584012bf01a52 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "024bc23e991c46bc813212a50b07f065" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 38309 } } peers { permanent_uuid: "8eb6c52ef06a4d5f97762c59705d0047" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 39975 } } peers { permanent_uuid: "813f1c245a4e49c499d584012bf01a52" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 38685 } }
I20260812 21:07:28.752090 8471 leader_election.cc:288] T d02b46993aa243c0a74732ae7ddd2db6 P 813f1c245a4e49c499d584012bf01a52 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 024bc23e991c46bc813212a50b07f065 (127.11.254.131:38309), 8eb6c52ef06a4d5f97762c59705d0047 (127.11.254.130:39975)
W20260812 21:07:28.752532 7782 leader_election.cc:341] T d02b46993aa243c0a74732ae7ddd2db6 P 813f1c245a4e49c499d584012bf01a52 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 024bc23e991c46bc813212a50b07f065 (127.11.254.131:38309): Not found: Tablet not found: d02b46993aa243c0a74732ae7ddd2db6
W20260812 21:07:28.752635 7788 leader_election.cc:341] T d02b46993aa243c0a74732ae7ddd2db6 P 813f1c245a4e49c499d584012bf01a52 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8eb6c52ef06a4d5f97762c59705d0047 (127.11.254.130:39975): Not found: Tablet not found: d02b46993aa243c0a74732ae7ddd2db6
I20260812 21:07:28.752676 7788 leader_election.cc:302] T d02b46993aa243c0a74732ae7ddd2db6 P 813f1c245a4e49c499d584012bf01a52 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 813f1c245a4e49c499d584012bf01a52; no voters: 024bc23e991c46bc813212a50b07f065, 8eb6c52ef06a4d5f97762c59705d0047
I20260812 21:07:28.752740 8471 raft_consensus.cc:2726] T d02b46993aa243c0a74732ae7ddd2db6 P 813f1c245a4e49c499d584012bf01a52 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 21:07:28.892318 8473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:28.892080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:38123 (local address 127.11.254.129:42891)
0812 21:07:28.892218 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0812 21:07:28.892266 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.190:38123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":68,"threads_started":1}
I20260812 21:07:28.943003 8477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:28.942702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:38123 (local address 127.11.254.130:38627)
0812 21:07:28.942895 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0812 21:07:28.942940 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.190:38123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":93,"threads_started":1}
I20260812 21:07:29.081375 8479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:29.081132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:38123 (local address 127.11.254.131:60285)
0812 21:07:29.081278 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0812 21:07:29.081324 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.190:38123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":70,"threads_started":1}
I20260812 21:07:29.893203 8480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:29.892937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:38123 (local address 127.11.254.129:50485)
0812 21:07:29.893103 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0812 21:07:29.893151 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.190:38123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":84,"threads_started":1}
I20260812 21:07:29.943686 8482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:29.943487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:38123 (local address 127.11.254.130:50471)
0812 21:07:29.943605 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0812 21:07:29.943637 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.190:38123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":65,"threads_started":1}
I20260812 21:07:30.082114 8483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:30.081880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:38123 (local address 127.11.254.131:46943)
0812 21:07:30.082024 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0812 21:07:30.082064 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.190:38123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":96,"threads_started":1}
I20260812 21:07:30.894152 8484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:30.893860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:38123 (local address 127.11.254.129:37305)
0812 21:07:30.894053 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0812 21:07:30.894101 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.190:38123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":126,"threads_started":1}
I20260812 21:07:30.944470 8486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:30.944165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:38123 (local address 127.11.254.130:52995)
0812 21:07:30.944344 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0812 21:07:30.944381 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.190:38123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":108,"threads_started":1}
I20260812 21:07:31.082966 8487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:31.082653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:38123 (local address 127.11.254.131:53795)
0812 21:07:31.082840 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0812 21:07:31.082890 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.190:38123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":104,"threads_started":1}
I20260812 21:07:31.895016 8489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:31.894748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:38123 (local address 127.11.254.129:40945)
0812 21:07:31.894915 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0812 21:07:31.894965 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.190:38123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":74,"threads_started":1}
I20260812 21:07:31.945269 8490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:31.945014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:38123 (local address 127.11.254.130:37711)
0812 21:07:31.945170 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0812 21:07:31.945219 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.190:38123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":70,"threads_started":1}
I20260812 21:07:32.083880 8492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:32.083619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:38123 (local address 127.11.254.131:56477)
0812 21:07:32.083781 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0812 21:07:32.083828 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.190:38123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":83,"threads_started":1}
I20260812 21:07:32.895931 8494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:32.895671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:38123 (local address 127.11.254.129:54347)
0812 21:07:32.895838 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0812 21:07:32.895881 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.190:38123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":83,"threads_started":1}
I20260812 21:07:32.946051 8495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:32.945793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:38123 (local address 127.11.254.130:45539)
0812 21:07:32.945926 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0812 21:07:32.945997 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.190:38123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":61,"threads_started":1}
I20260812 21:07:33.078553 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 7703
I20260812 21:07:33.088212 8496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:33.086563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:38123 (local address 127.11.254.131:35773)
0812 21:07:33.088108 (+ 1545us) negotiation.cc:107] Waiting for socket to connect
0812 21:07:33.088157 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.190:38123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1521,"thread_start_us":1423,"threads_started":1}
I20260812 21:07:33.092321 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 7970
I20260812 21:07:33.108621 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 8187
2026-08-12T21:07:33Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (58615 ms)
[----------] 30 tests from SecurityITest (245759 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568607354921-12282-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 21:07:33 dist-test-slave-vpvm krb5kdc[8500](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 21:07:33 dist-test-slave-vpvm krb5kdc[8500](info): set up 2 sockets
Aug 12 21:07:33 dist-test-slave-vpvm krb5kdc[8500](info): commencing operation
krb5kdc: starting...
W20260812 21:07:34.303536 12282 mini_kdc.cc:121] Time spent starting KDC: real 1.165s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568607354921-12282-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568607354921-12282-0/krb5kdc/test-user.keytab.
Aug 12 21:07:34 dist-test-slave-vpvm krb5kdc[8500](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568854, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T21:07:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T21:07:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.254.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.254.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.254.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.190.keytab.
Entry for principal kudu/127.11.254.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.190.keytab.
I20260812 21:07:34.429230 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568607354921-12282-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568607354921-12282-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568607354921-12282-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.254.190:42601
--webserver_interface=127.11.254.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.190.keytab
--principal=kudu/127.11.254.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.254.148:36891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.254.190:42601
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568607354921-12282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568607354921-12282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568607354921-12282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:07:34.511983 8519 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:07:34.512215 8519 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:07:34.512246 8519 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:07:34.513446 8519 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 21:07:34.513494 8519 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:07:34.513511 8519 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:07:34.513526 8519 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 21:07:34.513542 8519 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 21:07:34.515098 8519 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.254.148:36891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568607354921-12282-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568607354921-12282-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.254.190:42601
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.190.keytab
--principal=kudu/127.11.254.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.254.190:42601
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.254.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8519
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568607354921-12282-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:07:34.515352 8519 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:07:34.515578 8519 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:07:34.520960 8526 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:07:34.520979 8528 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:07:34.521030 8519 server_base.cc:1034] running on GCE node
W20260812 21:07:34.520960 8530 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:07:34.521456 8519 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:07:34.521736 8519 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:07:34.522895 8519 hybrid_clock.cc:648] HybridClock initialized: now 1786568854522875 us; error 31 us; skew 500 ppm
Aug 12 21:07:34 dist-test-slave-vpvm krb5kdc[8500](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568854, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:07:34.525568 8519 init.cc:377] Logged in from keytab as kudu/127.11.254.190@KRBTEST.COM (short username kudu)
I20260812 21:07:34.526046 8519 webserver.cc:460] Webserver started at http://127.11.254.190:44319/ using document root <none> and password file <none>
I20260812 21:07:34.526273 8519 fs_manager.cc:359] Metadata directory not provided
I20260812 21:07:34.526319 8519 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:07:34.526428 8519 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:07:34.527511 8519 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568607354921-12282-0/minicluster-data/master-0/data/instance:
uuid: "ff7f435573604945a810f2497612a6e7"
format_stamp: "Formatted at 2026-08-12 21:07:34 on dist-test-slave-vpvm"
server_key: "2794d1e810faed3734d428560f929c96"
server_key_iv: "6ed9e564f6ea831af751550d76ebc15e"
server_key_version: "encryptionkey@0"
I20260812 21:07:34.527863 8519 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568607354921-12282-0/minicluster-data/master-0/wal/instance:
uuid: "ff7f435573604945a810f2497612a6e7"
format_stamp: "Formatted at 2026-08-12 21:07:34 on dist-test-slave-vpvm"
server_key: "2794d1e810faed3734d428560f929c96"
server_key_iv: "6ed9e564f6ea831af751550d76ebc15e"
server_key_version: "encryptionkey@0"
I20260812 21:07:34.529264 8519 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 21:07:34.530082 8542 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:07:34.530252 8519 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 21:07:34.530341 8519 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568607354921-12282-0/minicluster-data/master-0/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568607354921-12282-0/minicluster-data/master-0/wal
uuid: "ff7f435573604945a810f2497612a6e7"
format_stamp: "Formatted at 2026-08-12 21:07:34 on dist-test-slave-vpvm"
server_key: "2794d1e810faed3734d428560f929c96"
server_key_iv: "6ed9e564f6ea831af751550d76ebc15e"
server_key_version: "encryptionkey@0"
I20260812 21:07:34.530402 8519 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568607354921-12282-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568607354921-12282-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568607354921-12282-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:07:34.541666 8519 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:07:34.543756 8519 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:07:34.543876 8519 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:07:34.547833 8519 rpc_server.cc:307] RPC server started. Bound to: 127.11.254.190:42601
I20260812 21:07:34.547884 8625 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.254.190:42601 every 8 connection(s)
I20260812 21:07:34.548229 8519 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
I20260812 21:07:34.549108 8626 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:07:34.551569 8626 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff7f435573604945a810f2497612a6e7: Bootstrap starting.
I20260812 21:07:34.552182 8626 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ff7f435573604945a810f2497612a6e7: Neither blocks nor log segments found. Creating new log.
I20260812 21:07:34.552459 8626 log.cc:824] T 00000000000000000000000000000000 P ff7f435573604945a810f2497612a6e7: Log is configured to *not* fsync() on all Append() calls
I20260812 21:07:34.553234 8626 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff7f435573604945a810f2497612a6e7: No bootstrap required, opened a new log
I20260812 21:07:34.554842 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 8519
I20260812 21:07:34.554957 12282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568607354921-12282-0/minicluster-data/master-0/wal/instance
I20260812 21:07:34.555140 12282 external_mini_cluster.cc:1428] Setting key 0dbefbc23ad0c71d1efe027c25b8b6bc
I20260812 21:07:34.555030 8626 raft_consensus.cc:348] T 00000000000000000000000000000000 P ff7f435573604945a810f2497612a6e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff7f435573604945a810f2497612a6e7" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 42601 } }
I20260812 21:07:34.555179 8626 raft_consensus.cc:374] T 00000000000000000000000000000000 P ff7f435573604945a810f2497612a6e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:07:34.555218 8626 raft_consensus.cc:729] T 00000000000000000000000000000000 P ff7f435573604945a810f2497612a6e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff7f435573604945a810f2497612a6e7, State: Initialized, Role: FOLLOWER
I20260812 21:07:34.555369 8626 consensus_queue.cc:260] T 00000000000000000000000000000000 P ff7f435573604945a810f2497612a6e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff7f435573604945a810f2497612a6e7" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 42601 } }
I20260812 21:07:34.555455 8626 raft_consensus.cc:388] T 00000000000000000000000000000000 P ff7f435573604945a810f2497612a6e7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 21:07:34.555487 8626 raft_consensus.cc:482] T 00000000000000000000000000000000 P ff7f435573604945a810f2497612a6e7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 21:07:34.555553 8626 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ff7f435573604945a810f2497612a6e7 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:07:34.556972 8626 raft_consensus.cc:504] T 00000000000000000000000000000000 P ff7f435573604945a810f2497612a6e7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff7f435573604945a810f2497612a6e7" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 42601 } }
I20260812 21:07:34.557081 8626 leader_election.cc:302] T 00000000000000000000000000000000 P ff7f435573604945a810f2497612a6e7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ff7f435573604945a810f2497612a6e7; no voters:
I20260812 21:07:34.557245 8626 leader_election.cc:288] T 00000000000000000000000000000000 P ff7f435573604945a810f2497612a6e7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 21:07:34.557283 8632 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ff7f435573604945a810f2497612a6e7 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:07:34.557493 8632 raft_consensus.cc:686] T 00000000000000000000000000000000 P ff7f435573604945a810f2497612a6e7 [term 1 LEADER]: Becoming Leader. State: Replica: ff7f435573604945a810f2497612a6e7, State: Running, Role: LEADER
I20260812 21:07:34.557520 8626 sys_catalog.cc:564] T 00000000000000000000000000000000 P ff7f435573604945a810f2497612a6e7 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 21:07:34.557641 8632 consensus_queue.cc:237] T 00000000000000000000000000000000 P ff7f435573604945a810f2497612a6e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff7f435573604945a810f2497612a6e7" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 42601 } }
I20260812 21:07:34.558120 8634 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff7f435573604945a810f2497612a6e7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ff7f435573604945a810f2497612a6e7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff7f435573604945a810f2497612a6e7" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 42601 } } }
I20260812 21:07:34.558204 8634 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff7f435573604945a810f2497612a6e7 [sys.catalog]: This master's current role is: LEADER
I20260812 21:07:34.558444 8635 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff7f435573604945a810f2497612a6e7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ff7f435573604945a810f2497612a6e7. Latest consensus state: current_term: 1 leader_uuid: "ff7f435573604945a810f2497612a6e7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff7f435573604945a810f2497612a6e7" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 42601 } } }
I20260812 21:07:34.558784 8635 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff7f435573604945a810f2497612a6e7 [sys.catalog]: This master's current role is: LEADER
W20260812 21:07:34.559397 8656 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ff7f435573604945a810f2497612a6e7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 21:07:34.559738 8656 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 21:07:34.559810 8644 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 21:07:34.560009 8644 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 21:07:34.561857 8644 catalog_manager.cc:1347] Generated new cluster ID: e7f5747575da4c4c87f1b41f2904de0d
I20260812 21:07:34.561903 8644 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 21:07:34 dist-test-slave-vpvm krb5kdc[8500](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568854, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.254.190@KRBTEST.COM
I20260812 21:07:34.567543 8631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:34.555651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50414 (local address 127.11.254.190:42601)
0812 21:07:34.555851 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0812 21:07:34.555861 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 21:07:34.555900 (+ 39us) server_negotiation.cc:407] Connection header received
0812 21:07:34.556015 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:07:34.556020 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:07:34.556065 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:07:34.556175 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:07:34.556998 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:34.558376 (+ 1378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:34.559549 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:34.560164 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:34.560282 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:07:34.563041 (+ 2759us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:07:34.563046 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:07:34.563046 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:07:34.563059 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:07:34.565695 (+ 2636us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:07:34.566153 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:07:34.566156 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:07:34.566156 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:07:34.566284 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:07:34.566496 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:07:34.566497 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:07:34.566497 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:07:34.566801 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:07:34.566941 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0812 21:07:34.567062 (+ 121us) server_negotiation.cc:299] Negotiation successful
0812 21:07:34.567301 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":106,"threads_started":1}
I20260812 21:07:34.579103 8644 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 21:07:34.579825 8644 catalog_manager.cc:1504] Loading token signing keys...
I20260812 21:07:34.595719 8644 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ff7f435573604945a810f2497612a6e7: Generated new TSK 0
I20260812 21:07:34.595964 8644 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.254.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.254.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.254.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.129.keytab.
Entry for principal kudu/127.11.254.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.129.keytab.
I20260812 21:07:34.627306 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568607354921-12282-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568607354921-12282-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568607354921-12282-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568607354921-12282-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.254.129:0
--local_ip_for_outbound_sockets=127.11.254.129
--webserver_interface=127.11.254.129
--webserver_port=0
--tserver_master_addrs=127.11.254.190:42601
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.129.keytab
--principal=kudu/127.11.254.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.254.148:36891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568607354921-12282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568607354921-12282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568607354921-12282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:07:34.710966 8665 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:07:34.711191 8665 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:07:34.711225 8665 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:07:34.712396 8665 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:07:34.712443 8665 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:07:34.712479 8665 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.254.129
I20260812 21:07:34.713984 8665 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.254.148:36891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568607354921-12282-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568607354921-12282-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.129.keytab
--principal=kudu/127.11.254.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.254.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568607354921-12282-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.254.129
--webserver_port=0
--tserver_master_addrs=127.11.254.190:42601
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8665
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.254.129
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568607354921-12282-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:07:34.714262 8665 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:07:34.714493 8665 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:07:34.719676 8675 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:07:34.719657 8674 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:07:34.719661 8677 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:07:34.719830 8665 server_base.cc:1034] running on GCE node
I20260812 21:07:34.720170 8665 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:07:34.720422 8665 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:07:34.721567 8665 hybrid_clock.cc:648] HybridClock initialized: now 1786568854721546 us; error 38 us; skew 500 ppm
Aug 12 21:07:34 dist-test-slave-vpvm krb5kdc[8500](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568854, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:07:34.724138 8665 init.cc:377] Logged in from keytab as kudu/127.11.254.129@KRBTEST.COM (short username kudu)
I20260812 21:07:34.724582 8665 webserver.cc:460] Webserver started at http://127.11.254.129:38321/ using document root <none> and password file <none>
I20260812 21:07:34.724813 8665 fs_manager.cc:359] Metadata directory not provided
I20260812 21:07:34.724864 8665 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:07:34.724992 8665 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:07:34.726146 8665 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568607354921-12282-0/minicluster-data/ts-0/data/instance:
uuid: "022a33213b32479face847c7df94069d"
format_stamp: "Formatted at 2026-08-12 21:07:34 on dist-test-slave-vpvm"
server_key: "b0b18fe4bdb11115c03c89d0be5bce10"
server_key_iv: "2437f670c72fd5c7f0a5ab64f11232b1"
server_key_version: "encryptionkey@0"
I20260812 21:07:34.726491 8665 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568607354921-12282-0/minicluster-data/ts-0/wal/instance:
uuid: "022a33213b32479face847c7df94069d"
format_stamp: "Formatted at 2026-08-12 21:07:34 on dist-test-slave-vpvm"
server_key: "b0b18fe4bdb11115c03c89d0be5bce10"
server_key_iv: "2437f670c72fd5c7f0a5ab64f11232b1"
server_key_version: "encryptionkey@0"
I20260812 21:07:34.727768 8665 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260812 21:07:34.728528 8688 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:07:34.728680 8665 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 21:07:34.728766 8665 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568607354921-12282-0/minicluster-data/ts-0/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568607354921-12282-0/minicluster-data/ts-0/wal
uuid: "022a33213b32479face847c7df94069d"
format_stamp: "Formatted at 2026-08-12 21:07:34 on dist-test-slave-vpvm"
server_key: "b0b18fe4bdb11115c03c89d0be5bce10"
server_key_iv: "2437f670c72fd5c7f0a5ab64f11232b1"
server_key_version: "encryptionkey@0"
I20260812 21:07:34.728826 8665 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568607354921-12282-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568607354921-12282-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568607354921-12282-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:07:34.738310 8665 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:07:34.740232 8665 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:07:34.740379 8665 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:07:34.740604 8665 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:07:34.740916 8665 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:07:34.740950 8665 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:07:34.740981 8665 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:07:34.741005 8665 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:07:34.746899 8665 rpc_server.cc:307] RPC server started. Bound to: 127.11.254.129:46793
I20260812 21:07:34.746956 8885 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.254.129:46793 every 8 connection(s)
I20260812 21:07:34.747289 8665 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568607354921-12282-0/minicluster-data/ts-0/data/info.pb
Aug 12 21:07:34 dist-test-slave-vpvm krb5kdc[8500](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568854, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.129@KRBTEST.COM for kudu/127.11.254.190@KRBTEST.COM
I20260812 21:07:34.753328 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 8665
I20260812 21:07:34.753415 12282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568607354921-12282-0/minicluster-data/ts-0/wal/instance
I20260812 21:07:34.753680 12282 external_mini_cluster.cc:1428] Setting key 9a9ba5ce979b3b3fea16a3fa9471e43a
I20260812 21:07:34.756522 8631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:34.748062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.129:43761 (local address 127.11.254.190:42601)
0812 21:07:34.748151 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0812 21:07:34.748153 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:07:34.748327 (+ 174us) server_negotiation.cc:407] Connection header received
0812 21:07:34.748839 (+ 512us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:07:34.748840 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:07:34.748855 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:07:34.748907 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:07:34.749895 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:34.750430 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:34.751414 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:34.751630 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:34.751686 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:07:34.753889 (+ 2203us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:07:34.753891 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:07:34.753891 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:07:34.753900 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:07:34.755265 (+ 1365us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:07:34.755646 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:07:34.755648 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:07:34.755648 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:07:34.755702 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:07:34.755919 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:07:34.755920 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:07:34.755920 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:07:34.756030 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:07:34.756088 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0812 21:07:34.756349 (+ 261us) server_negotiation.cc:299] Negotiation successful
0812 21:07:34.756434 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260812 21:07:34.756703 8889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:34.748027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:42601 (local address 127.11.254.129:43761)
0812 21:07:34.748205 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0812 21:07:34.748243 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 21:07:34.748757 (+ 514us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:07:34.748991 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:07:34.748995 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:07:34.749563 (+ 568us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:07:34.749799 (+ 236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:07:34.749805 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:07:34.750494 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:07:34.750495 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:07:34.751323 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:07:34.751326 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:07:34.751698 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:07:34.751699 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:07:34.751810 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:07:34.752276 (+ 466us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:07:34.752284 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:07:34.753787 (+ 1503us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:07:34.755344 (+ 1557us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:07:34.755352 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:07:34.755355 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:07:34.755578 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:07:34.755756 (+ 178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:07:34.755756 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:07:34.755757 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:07:34.755864 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:07:34.756086 (+ 222us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:07:34.756093 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:07:34.756232 (+ 139us) client_negotiation.cc:769] Sending connection context
0812 21:07:34.756318 (+ 86us) client_negotiation.cc:240] Negotiation successful
0812 21:07:34.756479 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":98,"threads_started":1}
I20260812 21:07:34.757155 8886 heartbeater.cc:347] Connected to a master server at 127.11.254.190:42601
I20260812 21:07:34.757247 8886 heartbeater.cc:464] Registering TS with master...
I20260812 21:07:34.757436 8886 heartbeater.cc:511] Master 127.11.254.190:42601 requested a full tablet report, sending...
I20260812 21:07:34.757988 8572 ts_manager.cc:194] Registered new tserver with Master: 022a33213b32479face847c7df94069d (127.11.254.129:46793)
I20260812 21:07:34.758641 8572 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.254.129@KRBTEST.COM'} at 127.11.254.129:43761
WARNING: no policy specified for kudu/127.11.254.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.254.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.254.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.130.keytab.
Entry for principal kudu/127.11.254.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.130.keytab.
I20260812 21:07:34.780424 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568607354921-12282-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568607354921-12282-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568607354921-12282-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568607354921-12282-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.254.130:0
--local_ip_for_outbound_sockets=127.11.254.130
--webserver_interface=127.11.254.130
--webserver_port=0
--tserver_master_addrs=127.11.254.190:42601
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.130.keytab
--principal=kudu/127.11.254.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.254.148:36891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568607354921-12282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568607354921-12282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568607354921-12282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:07:34.863425 8892 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:07:34.863632 8892 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:07:34.863664 8892 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:07:34.864776 8892 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:07:34.864816 8892 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:07:34.864850 8892 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.254.130
I20260812 21:07:34.866282 8892 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.254.148:36891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568607354921-12282-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568607354921-12282-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.130.keytab
--principal=kudu/127.11.254.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.254.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568607354921-12282-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.254.130
--webserver_port=0
--tserver_master_addrs=127.11.254.190:42601
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8892
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.254.130
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568607354921-12282-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:07:34.866518 8892 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:07:34.866729 8892 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:07:34.871941 8906 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:07:34.871970 8904 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:07:34.871973 8903 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:07:34.872016 8892 server_base.cc:1034] running on GCE node
I20260812 21:07:34.872357 8892 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:07:34.872613 8892 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:07:34.873762 8892 hybrid_clock.cc:648] HybridClock initialized: now 1786568854873750 us; error 28 us; skew 500 ppm
Aug 12 21:07:34 dist-test-slave-vpvm krb5kdc[8500](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568854, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:07:34.876372 8892 init.cc:377] Logged in from keytab as kudu/127.11.254.130@KRBTEST.COM (short username kudu)
I20260812 21:07:34.876820 8892 webserver.cc:460] Webserver started at http://127.11.254.130:43473/ using document root <none> and password file <none>
I20260812 21:07:34.877075 8892 fs_manager.cc:359] Metadata directory not provided
I20260812 21:07:34.877134 8892 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:07:34.877260 8892 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:07:34.878408 8892 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568607354921-12282-0/minicluster-data/ts-1/data/instance:
uuid: "3f2bca5a4eae4ae997476203d5bb9f1d"
format_stamp: "Formatted at 2026-08-12 21:07:34 on dist-test-slave-vpvm"
server_key: "ee8b714dfab57453c64448cfd5a105af"
server_key_iv: "508c9d54514bb53c3578d327cd7d3b8b"
server_key_version: "encryptionkey@0"
I20260812 21:07:34.878756 8892 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568607354921-12282-0/minicluster-data/ts-1/wal/instance:
uuid: "3f2bca5a4eae4ae997476203d5bb9f1d"
format_stamp: "Formatted at 2026-08-12 21:07:34 on dist-test-slave-vpvm"
server_key: "ee8b714dfab57453c64448cfd5a105af"
server_key_iv: "508c9d54514bb53c3578d327cd7d3b8b"
server_key_version: "encryptionkey@0"
I20260812 21:07:34.880112 8892 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260812 21:07:34.880901 8919 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:07:34.881090 8892 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 21:07:34.881187 8892 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568607354921-12282-0/minicluster-data/ts-1/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568607354921-12282-0/minicluster-data/ts-1/wal
uuid: "3f2bca5a4eae4ae997476203d5bb9f1d"
format_stamp: "Formatted at 2026-08-12 21:07:34 on dist-test-slave-vpvm"
server_key: "ee8b714dfab57453c64448cfd5a105af"
server_key_iv: "508c9d54514bb53c3578d327cd7d3b8b"
server_key_version: "encryptionkey@0"
I20260812 21:07:34.881255 8892 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568607354921-12282-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568607354921-12282-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568607354921-12282-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:07:34.898945 8892 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:07:34.901189 8892 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:07:34.901345 8892 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:07:34.901583 8892 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:07:34.901899 8892 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:07:34.901934 8892 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:07:34.901990 8892 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:07:34.902016 8892 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:07:34.907661 8892 rpc_server.cc:307] RPC server started. Bound to: 127.11.254.130:35807
I20260812 21:07:34.907719 9094 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.254.130:35807 every 8 connection(s)
I20260812 21:07:34.908035 8892 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568607354921-12282-0/minicluster-data/ts-1/data/info.pb
Aug 12 21:07:34 dist-test-slave-vpvm krb5kdc[8500](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568854, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.130@KRBTEST.COM for kudu/127.11.254.190@KRBTEST.COM
I20260812 21:07:34.916296 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 8892
I20260812 21:07:34.916393 12282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568607354921-12282-0/minicluster-data/ts-1/wal/instance
I20260812 21:07:34.916607 12282 external_mini_cluster.cc:1428] Setting key c4a15b67d09f5e79ec6e62e5ff8b2f85
I20260812 21:07:34.917132 8631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:34.908721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.130:51579 (local address 127.11.254.190:42601)
0812 21:07:34.908782 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0812 21:07:34.908784 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:07:34.908964 (+ 180us) server_negotiation.cc:407] Connection header received
0812 21:07:34.909485 (+ 521us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:07:34.909487 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:07:34.909500 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:07:34.909553 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:07:34.910570 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:34.911094 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:34.912088 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:34.912292 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:34.912350 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:07:34.914423 (+ 2073us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:07:34.914425 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:07:34.914425 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:07:34.914433 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:07:34.915690 (+ 1257us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:07:34.916080 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:07:34.916081 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:07:34.916082 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:07:34.916155 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:07:34.916413 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:07:34.916414 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:07:34.916414 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:07:34.916547 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:07:34.916607 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0812 21:07:34.916927 (+ 320us) server_negotiation.cc:299] Negotiation successful
0812 21:07:34.917018 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260812 21:07:34.917339 9099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:34.908695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:42601 (local address 127.11.254.130:51579)
0812 21:07:34.908846 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0812 21:07:34.908883 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0812 21:07:34.909381 (+ 498us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:07:34.909678 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:07:34.909681 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:07:34.910261 (+ 580us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:07:34.910465 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:07:34.910470 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:07:34.911184 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:07:34.911185 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:07:34.911996 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:07:34.911998 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:07:34.912365 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:07:34.912366 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:07:34.912484 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:07:34.912973 (+ 489us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:07:34.912981 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:07:34.914332 (+ 1351us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:07:34.915767 (+ 1435us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:07:34.915775 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:07:34.915778 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:07:34.916013 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:07:34.916209 (+ 196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:07:34.916210 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:07:34.916210 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:07:34.916316 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:07:34.916605 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:07:34.916614 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:07:34.916781 (+ 167us) client_negotiation.cc:769] Sending connection context
0812 21:07:34.916887 (+ 106us) client_negotiation.cc:240] Negotiation successful
0812 21:07:34.917060 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":79,"threads_started":1}
I20260812 21:07:34.917750 9097 heartbeater.cc:347] Connected to a master server at 127.11.254.190:42601
I20260812 21:07:34.917841 9097 heartbeater.cc:464] Registering TS with master...
I20260812 21:07:34.918080 9097 heartbeater.cc:511] Master 127.11.254.190:42601 requested a full tablet report, sending...
I20260812 21:07:34.918463 8572 ts_manager.cc:194] Registered new tserver with Master: 3f2bca5a4eae4ae997476203d5bb9f1d (127.11.254.130:35807)
I20260812 21:07:34.918843 8572 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.254.130@KRBTEST.COM'} at 127.11.254.130:51579
WARNING: no policy specified for kudu/127.11.254.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.254.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.254.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.131.keytab.
Entry for principal kudu/127.11.254.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.131.keytab.
I20260812 21:07:34.942926 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568607354921-12282-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568607354921-12282-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568607354921-12282-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568607354921-12282-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.254.131:0
--local_ip_for_outbound_sockets=127.11.254.131
--webserver_interface=127.11.254.131
--webserver_port=0
--tserver_master_addrs=127.11.254.190:42601
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.131.keytab
--principal=kudu/127.11.254.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.254.148:36891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568607354921-12282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568607354921-12282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568607354921-12282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:07:35.025472 9103 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:07:35.025712 9103 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:07:35.025745 9103 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:07:35.026999 9103 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:07:35.027058 9103 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:07:35.027098 9103 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.254.131
I20260812 21:07:35.028599 9103 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.254.148:36891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568607354921-12282-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568607354921-12282-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.131.keytab
--principal=kudu/127.11.254.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.254.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568607354921-12282-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.254.131
--webserver_port=0
--tserver_master_addrs=127.11.254.190:42601
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9103
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.254.131
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568607354921-12282-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:07:35.028846 9103 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:07:35.029080 9103 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:07:35.034209 9110 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:07:35.034209 9120 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:07:35.034263 9103 server_base.cc:1034] running on GCE node
W20260812 21:07:35.034323 9112 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:07:35.034662 9103 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:07:35.034869 9103 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:07:35.035995 9103 hybrid_clock.cc:648] HybridClock initialized: now 1786568855035977 us; error 33 us; skew 500 ppm
Aug 12 21:07:35 dist-test-slave-vpvm krb5kdc[8500](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568855, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:07:35.038517 9103 init.cc:377] Logged in from keytab as kudu/127.11.254.131@KRBTEST.COM (short username kudu)
I20260812 21:07:35.038970 9103 webserver.cc:460] Webserver started at http://127.11.254.131:34135/ using document root <none> and password file <none>
I20260812 21:07:35.039163 9103 fs_manager.cc:359] Metadata directory not provided
I20260812 21:07:35.039204 9103 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:07:35.039295 9103 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:07:35.040364 9103 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568607354921-12282-0/minicluster-data/ts-2/data/instance:
uuid: "cf288bd7fbf7423a8b982d5e57a2ed53"
format_stamp: "Formatted at 2026-08-12 21:07:35 on dist-test-slave-vpvm"
server_key: "fa155384897a6bfc2a0a0fd91e6e7cf0"
server_key_iv: "772f9bbd504faee193a81af597d5a5f1"
server_key_version: "encryptionkey@0"
I20260812 21:07:35.040679 9103 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568607354921-12282-0/minicluster-data/ts-2/wal/instance:
uuid: "cf288bd7fbf7423a8b982d5e57a2ed53"
format_stamp: "Formatted at 2026-08-12 21:07:35 on dist-test-slave-vpvm"
server_key: "fa155384897a6bfc2a0a0fd91e6e7cf0"
server_key_iv: "772f9bbd504faee193a81af597d5a5f1"
server_key_version: "encryptionkey@0"
I20260812 21:07:35.041859 9103 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260812 21:07:35.042551 9128 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:07:35.042701 9103 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 21:07:35.042780 9103 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568607354921-12282-0/minicluster-data/ts-2/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568607354921-12282-0/minicluster-data/ts-2/wal
uuid: "cf288bd7fbf7423a8b982d5e57a2ed53"
format_stamp: "Formatted at 2026-08-12 21:07:35 on dist-test-slave-vpvm"
server_key: "fa155384897a6bfc2a0a0fd91e6e7cf0"
server_key_iv: "772f9bbd504faee193a81af597d5a5f1"
server_key_version: "encryptionkey@0"
I20260812 21:07:35.042835 9103 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568607354921-12282-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568607354921-12282-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568607354921-12282-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:07:35.067988 9103 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:07:35.070155 9103 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:07:35.070300 9103 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:07:35.070545 9103 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:07:35.070868 9103 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:07:35.070902 9103 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:07:35.070925 9103 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:07:35.070948 9103 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:07:35.076551 9103 rpc_server.cc:307] RPC server started. Bound to: 127.11.254.131:40297
I20260812 21:07:35.076648 9312 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.254.131:40297 every 8 connection(s)
I20260812 21:07:35.076934 9103 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568607354921-12282-0/minicluster-data/ts-2/data/info.pb
I20260812 21:07:35.078871 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 9103
I20260812 21:07:35.078979 12282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568607354921-12282-0/minicluster-data/ts-2/wal/instance
I20260812 21:07:35.079182 12282 external_mini_cluster.cc:1428] Setting key d03f79aea35041d6002025f3344456da
Aug 12 21:07:35 dist-test-slave-vpvm krb5kdc[8500](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568855, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.131@KRBTEST.COM for kudu/127.11.254.190@KRBTEST.COM
I20260812 21:07:35.086762 8631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:35.077739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.131:59489 (local address 127.11.254.190:42601)
0812 21:07:35.077807 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0812 21:07:35.077809 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:07:35.078019 (+ 210us) server_negotiation.cc:407] Connection header received
0812 21:07:35.078516 (+ 497us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:07:35.078517 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:07:35.078528 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:07:35.078578 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:07:35.079964 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:35.080481 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:35.081470 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:35.081661 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:35.081717 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:07:35.084049 (+ 2332us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:07:35.084050 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:07:35.084050 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:07:35.084058 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:07:35.085410 (+ 1352us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:07:35.085802 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:07:35.085804 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:07:35.085804 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:07:35.085857 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:07:35.086116 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:07:35.086117 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:07:35.086117 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:07:35.086236 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:07:35.086293 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0812 21:07:35.086572 (+ 279us) server_negotiation.cc:299] Negotiation successful
0812 21:07:35.086669 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260812 21:07:35.086941 9317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:35.077708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:42601 (local address 127.11.254.131:59489)
0812 21:07:35.077876 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0812 21:07:35.077912 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0812 21:07:35.078436 (+ 524us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:07:35.078633 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:07:35.078636 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:07:35.079424 (+ 788us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:07:35.079743 (+ 319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:07:35.079751 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:07:35.080582 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:07:35.080583 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:07:35.081386 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:07:35.081388 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:07:35.081729 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:07:35.081730 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:07:35.081862 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:07:35.082360 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:07:35.082369 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:07:35.083936 (+ 1567us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:07:35.085491 (+ 1555us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:07:35.085499 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:07:35.085501 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:07:35.085732 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:07:35.085910 (+ 178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:07:35.085911 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:07:35.085911 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:07:35.086057 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:07:35.086292 (+ 235us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:07:35.086300 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:07:35.086440 (+ 140us) client_negotiation.cc:769] Sending connection context
0812 21:07:35.086525 (+ 85us) client_negotiation.cc:240] Negotiation successful
0812 21:07:35.086703 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":90,"threads_started":1}
I20260812 21:07:35.087390 9313 heartbeater.cc:347] Connected to a master server at 127.11.254.190:42601
I20260812 21:07:35.087482 9313 heartbeater.cc:464] Registering TS with master...
I20260812 21:07:35.087641 9313 heartbeater.cc:511] Master 127.11.254.190:42601 requested a full tablet report, sending...
I20260812 21:07:35.087993 8572 ts_manager.cc:194] Registered new tserver with Master: cf288bd7fbf7423a8b982d5e57a2ed53 (127.11.254.131:40297)
I20260812 21:07:35.088420 8572 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.254.131@KRBTEST.COM'} at 127.11.254.131:59489
I20260812 21:07:35.090795 12282 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 21:07:35.098089 8631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:35.092097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50416 (local address 127.11.254.190:42601)
0812 21:07:35.092150 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0812 21:07:35.092151 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 21:07:35.092237 (+ 86us) server_negotiation.cc:407] Connection header received
0812 21:07:35.092285 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:07:35.092286 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:07:35.092293 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:07:35.092340 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:07:35.092826 (+ 486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:35.093299 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:35.094163 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:35.094369 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:35.094433 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:07:35.095732 (+ 1299us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:07:35.095734 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:07:35.095734 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:07:35.095742 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:07:35.097064 (+ 1322us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:07:35.097362 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:07:35.097363 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:07:35.097363 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:07:35.097408 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:07:35.097624 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:07:35.097624 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:07:35.097625 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:07:35.097770 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:07:35.097843 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 21:07:35.097926 (+ 83us) server_negotiation.cc:299] Negotiation successful
0812 21:07:35.098019 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260812 21:07:35.099699 8572 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50416:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 21:07:35.101248 8572 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 21:07:35.107607 9333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:35.103752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49440 (local address 127.11.254.130:35807)
0812 21:07:35.103931 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0812 21:07:35.103935 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 21:07:35.104031 (+ 96us) server_negotiation.cc:407] Connection header received
0812 21:07:35.104112 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:07:35.104113 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:07:35.104126 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:07:35.104211 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:07:35.104635 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:35.105313 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:35.106626 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:35.107159 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:35.107232 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:07:35.107297 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0812 21:07:35.107480 (+ 183us) server_negotiation.cc:299] Negotiation successful
0812 21:07:35.107523 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":79,"threads_started":1}
I20260812 21:07:35.107591 9331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:35.103505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.130:35807 (local address 127.0.0.1:49440)
0812 21:07:35.103926 (+ 421us) negotiation.cc:107] Waiting for socket to connect
0812 21:07:35.103967 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 21:07:35.104040 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:07:35.104231 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:07:35.104235 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:07:35.104238 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:07:35.104510 (+ 272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:07:35.104513 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:07:35.105406 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:07:35.105407 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:07:35.106541 (+ 1134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:07:35.106544 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:07:35.107240 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:07:35.107241 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:07:35.107355 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:07:35.107383 (+ 28us) client_negotiation.cc:769] Sending connection context
0812 21:07:35.107444 (+ 61us) client_negotiation.cc:240] Negotiation successful
0812 21:07:35.107510 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":386,"thread_start_us":143,"threads_started":1}
I20260812 21:07:35.108176 9328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:35.103520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.131:40297 (local address 127.0.0.1:50380)
0812 21:07:35.104214 (+ 694us) negotiation.cc:107] Waiting for socket to connect
0812 21:07:35.104222 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0812 21:07:35.104263 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:07:35.104524 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:07:35.104525 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:07:35.104527 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:07:35.104565 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:07:35.104565 (+ 0us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:07:35.105809 (+ 1244us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:07:35.105810 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:07:35.106632 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:07:35.106635 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:07:35.107611 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:07:35.107612 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:07:35.107983 (+ 371us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:07:35.107989 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 21:07:35.108049 (+ 60us) client_negotiation.cc:240] Negotiation successful
0812 21:07:35.108103 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":687,"mutex_wait_us":32,"thread_start_us":89,"threads_started":1}
I20260812 21:07:35.108214 9334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:35.103743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50380 (local address 127.11.254.131:40297)
0812 21:07:35.103935 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0812 21:07:35.103940 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 21:07:35.104313 (+ 373us) server_negotiation.cc:407] Connection header received
0812 21:07:35.104358 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:07:35.104360 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:07:35.104376 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:07:35.104443 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:07:35.104627 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:35.105706 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:35.106718 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:35.107458 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:35.107523 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:07:35.107575 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0812 21:07:35.108091 (+ 516us) server_negotiation.cc:299] Negotiation successful
0812 21:07:35.108138 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":88,"threads_started":1}
I20260812 21:07:35.108349 8994 tablet_service.cc:1467] Processing CreateTablet for tablet 4c44605f34e84bb4930f84f51f063c6c (DEFAULT_TABLE table=test-table [id=7c2cb47840664ba0ba77d2d7089f2b58]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:07:35.108409 9329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:35.103505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.129:46793 (local address 127.0.0.1:47362)
0812 21:07:35.104128 (+ 623us) negotiation.cc:107] Waiting for socket to connect
0812 21:07:35.104135 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0812 21:07:35.104169 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:07:35.104357 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:07:35.104359 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:07:35.104361 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:07:35.104437 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:07:35.104438 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:07:35.105799 (+ 1361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:07:35.105801 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:07:35.107255 (+ 1454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:07:35.107258 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:07:35.108204 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:07:35.108205 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:07:35.108267 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:07:35.108271 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 21:07:35.108320 (+ 49us) client_negotiation.cc:240] Negotiation successful
0812 21:07:35.108353 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":609,"mutex_wait_us":60,"thread_start_us":49,"threads_started":1}
I20260812 21:07:35.108733 8994 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4c44605f34e84bb4930f84f51f063c6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:07:35.108821 9332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:35.103726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47362 (local address 127.11.254.129:46793)
0812 21:07:35.103911 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0812 21:07:35.103917 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 21:07:35.104205 (+ 288us) server_negotiation.cc:407] Connection header received
0812 21:07:35.104252 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:07:35.104254 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:07:35.104268 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:07:35.104354 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:07:35.104577 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:35.105688 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:35.107354 (+ 1666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:35.108061 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:35.108120 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:07:35.108170 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0812 21:07:35.108474 (+ 304us) server_negotiation.cc:299] Negotiation successful
0812 21:07:35.108738 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":85,"threads_started":1}
I20260812 21:07:35.109105 9201 tablet_service.cc:1467] Processing CreateTablet for tablet 4c44605f34e84bb4930f84f51f063c6c (DEFAULT_TABLE table=test-table [id=7c2cb47840664ba0ba77d2d7089f2b58]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:07:35.109423 9201 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4c44605f34e84bb4930f84f51f063c6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:07:35.109452 8769 tablet_service.cc:1467] Processing CreateTablet for tablet 4c44605f34e84bb4930f84f51f063c6c (DEFAULT_TABLE table=test-table [id=7c2cb47840664ba0ba77d2d7089f2b58]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:07:35.109761 8769 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4c44605f34e84bb4930f84f51f063c6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:07:35.111363 9335 tablet_bootstrap.cc:492] T 4c44605f34e84bb4930f84f51f063c6c P 3f2bca5a4eae4ae997476203d5bb9f1d: Bootstrap starting.
I20260812 21:07:35.111960 9335 tablet_bootstrap.cc:654] T 4c44605f34e84bb4930f84f51f063c6c P 3f2bca5a4eae4ae997476203d5bb9f1d: Neither blocks nor log segments found. Creating new log.
I20260812 21:07:35.112351 9335 log.cc:824] T 4c44605f34e84bb4930f84f51f063c6c P 3f2bca5a4eae4ae997476203d5bb9f1d: Log is configured to *not* fsync() on all Append() calls
I20260812 21:07:35.112653 9340 tablet_bootstrap.cc:492] T 4c44605f34e84bb4930f84f51f063c6c P cf288bd7fbf7423a8b982d5e57a2ed53: Bootstrap starting.
I20260812 21:07:35.112991 9341 tablet_bootstrap.cc:492] T 4c44605f34e84bb4930f84f51f063c6c P 022a33213b32479face847c7df94069d: Bootstrap starting.
I20260812 21:07:35.113255 9340 tablet_bootstrap.cc:654] T 4c44605f34e84bb4930f84f51f063c6c P cf288bd7fbf7423a8b982d5e57a2ed53: Neither blocks nor log segments found. Creating new log.
I20260812 21:07:35.113539 9341 tablet_bootstrap.cc:654] T 4c44605f34e84bb4930f84f51f063c6c P 022a33213b32479face847c7df94069d: Neither blocks nor log segments found. Creating new log.
I20260812 21:07:35.113683 9340 log.cc:824] T 4c44605f34e84bb4930f84f51f063c6c P cf288bd7fbf7423a8b982d5e57a2ed53: Log is configured to *not* fsync() on all Append() calls
I20260812 21:07:35.114102 9341 log.cc:824] T 4c44605f34e84bb4930f84f51f063c6c P 022a33213b32479face847c7df94069d: Log is configured to *not* fsync() on all Append() calls
I20260812 21:07:35.114471 9335 tablet_bootstrap.cc:492] T 4c44605f34e84bb4930f84f51f063c6c P 3f2bca5a4eae4ae997476203d5bb9f1d: No bootstrap required, opened a new log
I20260812 21:07:35.114547 9335 ts_tablet_manager.cc:1397] T 4c44605f34e84bb4930f84f51f063c6c P 3f2bca5a4eae4ae997476203d5bb9f1d: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260812 21:07:35.114821 9341 tablet_bootstrap.cc:492] T 4c44605f34e84bb4930f84f51f063c6c P 022a33213b32479face847c7df94069d: No bootstrap required, opened a new log
I20260812 21:07:35.114888 9341 ts_tablet_manager.cc:1397] T 4c44605f34e84bb4930f84f51f063c6c P 022a33213b32479face847c7df94069d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 21:07:35.116539 9335 raft_consensus.cc:348] T 4c44605f34e84bb4930f84f51f063c6c P 3f2bca5a4eae4ae997476203d5bb9f1d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "022a33213b32479face847c7df94069d" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 46793 } } peers { permanent_uuid: "3f2bca5a4eae4ae997476203d5bb9f1d" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 35807 } } peers { permanent_uuid: "cf288bd7fbf7423a8b982d5e57a2ed53" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 40297 } }
I20260812 21:07:35.116752 9335 raft_consensus.cc:374] T 4c44605f34e84bb4930f84f51f063c6c P 3f2bca5a4eae4ae997476203d5bb9f1d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:07:35.116804 9335 raft_consensus.cc:729] T 4c44605f34e84bb4930f84f51f063c6c P 3f2bca5a4eae4ae997476203d5bb9f1d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f2bca5a4eae4ae997476203d5bb9f1d, State: Initialized, Role: FOLLOWER
I20260812 21:07:35.116811 9341 raft_consensus.cc:348] T 4c44605f34e84bb4930f84f51f063c6c P 022a33213b32479face847c7df94069d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "022a33213b32479face847c7df94069d" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 46793 } } peers { permanent_uuid: "3f2bca5a4eae4ae997476203d5bb9f1d" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 35807 } } peers { permanent_uuid: "cf288bd7fbf7423a8b982d5e57a2ed53" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 40297 } }
I20260812 21:07:35.116931 9335 consensus_queue.cc:260] T 4c44605f34e84bb4930f84f51f063c6c P 3f2bca5a4eae4ae997476203d5bb9f1d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "022a33213b32479face847c7df94069d" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 46793 } } peers { permanent_uuid: "3f2bca5a4eae4ae997476203d5bb9f1d" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 35807 } } peers { permanent_uuid: "cf288bd7fbf7423a8b982d5e57a2ed53" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 40297 } }
I20260812 21:07:35.116971 9341 raft_consensus.cc:374] T 4c44605f34e84bb4930f84f51f063c6c P 022a33213b32479face847c7df94069d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:07:35.117010 9341 raft_consensus.cc:729] T 4c44605f34e84bb4930f84f51f063c6c P 022a33213b32479face847c7df94069d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 022a33213b32479face847c7df94069d, State: Initialized, Role: FOLLOWER
I20260812 21:07:35.117147 9341 consensus_queue.cc:260] T 4c44605f34e84bb4930f84f51f063c6c P 022a33213b32479face847c7df94069d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "022a33213b32479face847c7df94069d" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 46793 } } peers { permanent_uuid: "3f2bca5a4eae4ae997476203d5bb9f1d" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 35807 } } peers { permanent_uuid: "cf288bd7fbf7423a8b982d5e57a2ed53" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 40297 } }
I20260812 21:07:35.117239 9335 ts_tablet_manager.cc:1428] T 4c44605f34e84bb4930f84f51f063c6c P 3f2bca5a4eae4ae997476203d5bb9f1d: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 21:07:35.117264 9097 heartbeater.cc:503] Master 127.11.254.190:42601 was elected leader, sending a full tablet report...
I20260812 21:07:35.117412 9341 ts_tablet_manager.cc:1428] T 4c44605f34e84bb4930f84f51f063c6c P 022a33213b32479face847c7df94069d: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260812 21:07:35.117530 8886 heartbeater.cc:503] Master 127.11.254.190:42601 was elected leader, sending a full tablet report...
I20260812 21:07:35.117697 9340 tablet_bootstrap.cc:492] T 4c44605f34e84bb4930f84f51f063c6c P cf288bd7fbf7423a8b982d5e57a2ed53: No bootstrap required, opened a new log
I20260812 21:07:35.117765 9340 ts_tablet_manager.cc:1397] T 4c44605f34e84bb4930f84f51f063c6c P cf288bd7fbf7423a8b982d5e57a2ed53: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260812 21:07:35.119445 9340 raft_consensus.cc:348] T 4c44605f34e84bb4930f84f51f063c6c P cf288bd7fbf7423a8b982d5e57a2ed53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "022a33213b32479face847c7df94069d" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 46793 } } peers { permanent_uuid: "3f2bca5a4eae4ae997476203d5bb9f1d" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 35807 } } peers { permanent_uuid: "cf288bd7fbf7423a8b982d5e57a2ed53" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 40297 } }
I20260812 21:07:35.119587 9340 raft_consensus.cc:374] T 4c44605f34e84bb4930f84f51f063c6c P cf288bd7fbf7423a8b982d5e57a2ed53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:07:35.119609 9340 raft_consensus.cc:729] T 4c44605f34e84bb4930f84f51f063c6c P cf288bd7fbf7423a8b982d5e57a2ed53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf288bd7fbf7423a8b982d5e57a2ed53, State: Initialized, Role: FOLLOWER
I20260812 21:07:35.119705 9340 consensus_queue.cc:260] T 4c44605f34e84bb4930f84f51f063c6c P cf288bd7fbf7423a8b982d5e57a2ed53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "022a33213b32479face847c7df94069d" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 46793 } } peers { permanent_uuid: "3f2bca5a4eae4ae997476203d5bb9f1d" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 35807 } } peers { permanent_uuid: "cf288bd7fbf7423a8b982d5e57a2ed53" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 40297 } }
I20260812 21:07:35.119954 9340 ts_tablet_manager.cc:1428] T 4c44605f34e84bb4930f84f51f063c6c P cf288bd7fbf7423a8b982d5e57a2ed53: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260812 21:07:35.120014 9313 heartbeater.cc:503] Master 127.11.254.190:42601 was elected leader, sending a full tablet report...
W20260812 21:07:35.158649 9098 tablet.cc:2369] T 4c44605f34e84bb4930f84f51f063c6c P 3f2bca5a4eae4ae997476203d5bb9f1d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 21:07:35.248129 8887 tablet.cc:2369] T 4c44605f34e84bb4930f84f51f063c6c P 022a33213b32479face847c7df94069d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 21:07:35.327594 9314 tablet.cc:2369] T 4c44605f34e84bb4930f84f51f063c6c P cf288bd7fbf7423a8b982d5e57a2ed53: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 21:07:35.345721 9344 raft_consensus.cc:482] T 4c44605f34e84bb4930f84f51f063c6c P 3f2bca5a4eae4ae997476203d5bb9f1d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 21:07:35.345872 9344 raft_consensus.cc:504] T 4c44605f34e84bb4930f84f51f063c6c P 3f2bca5a4eae4ae997476203d5bb9f1d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "022a33213b32479face847c7df94069d" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 46793 } } peers { permanent_uuid: "3f2bca5a4eae4ae997476203d5bb9f1d" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 35807 } } peers { permanent_uuid: "cf288bd7fbf7423a8b982d5e57a2ed53" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 40297 } }
I20260812 21:07:35.346268 9344 leader_election.cc:288] T 4c44605f34e84bb4930f84f51f063c6c P 3f2bca5a4eae4ae997476203d5bb9f1d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 022a33213b32479face847c7df94069d (127.11.254.129:46793), cf288bd7fbf7423a8b982d5e57a2ed53 (127.11.254.131:40297)
I20260812 21:07:35.349306 9099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:35.346482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.129:46793 (local address 127.11.254.130:55383)
0812 21:07:35.346541 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0812 21:07:35.346553 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 21:07:35.346600 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:07:35.346780 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:07:35.346780 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:07:35.346782 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:07:35.346879 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:07:35.346881 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:07:35.347667 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:07:35.347668 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:07:35.348461 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:07:35.348463 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:07:35.349075 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:07:35.349076 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:07:35.349153 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:07:35.349157 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 21:07:35.349207 (+ 50us) client_negotiation.cc:240] Negotiation successful
0812 21:07:35.349247 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260812 21:07:35.349332 9332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:35.346535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.130:55383 (local address 127.11.254.129:46793)
0812 21:07:35.346584 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0812 21:07:35.346586 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:07:35.346600 (+ 14us) server_negotiation.cc:407] Connection header received
0812 21:07:35.346668 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:07:35.346669 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:07:35.346682 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:07:35.346760 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:07:35.347002 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:35.347584 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:35.348606 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:35.348997 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:35.349052 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:07:35.349060 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 21:07:35.349220 (+ 160us) server_negotiation.cc:299] Negotiation successful
0812 21:07:35.349260 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260812 21:07:35.349448 9348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:35.346497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.131:40297 (local address 127.11.254.130:38619)
0812 21:07:35.346740 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0812 21:07:35.346750 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 21:07:35.346812 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:07:35.346978 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:07:35.346979 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:07:35.346982 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:07:35.347083 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:07:35.347085 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:07:35.347815 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:07:35.347816 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:07:35.348608 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:07:35.348611 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:07:35.349187 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:07:35.349188 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:07:35.349283 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:07:35.349287 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 21:07:35.349342 (+ 55us) client_negotiation.cc:240] Negotiation successful
0812 21:07:35.349377 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":130,"threads_started":1}
I20260812 21:07:35.349464 9334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:35.346570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.130:38619 (local address 127.11.254.131:40297)
0812 21:07:35.346619 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0812 21:07:35.346621 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:07:35.346825 (+ 204us) server_negotiation.cc:407] Connection header received
0812 21:07:35.346855 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:07:35.346857 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:07:35.346867 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:07:35.346926 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:07:35.347188 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:35.347736 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:35.348689 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:35.349108 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:35.349177 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:07:35.349185 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 21:07:35.349356 (+ 171us) server_negotiation.cc:299] Negotiation successful
0812 21:07:35.349405 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260812 21:07:35.349689 8797 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c44605f34e84bb4930f84f51f063c6c" candidate_uuid: "3f2bca5a4eae4ae997476203d5bb9f1d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "022a33213b32479face847c7df94069d" is_pre_election: true
I20260812 21:07:35.349861 8797 raft_consensus.cc:2454] T 4c44605f34e84bb4930f84f51f063c6c P 022a33213b32479face847c7df94069d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f2bca5a4eae4ae997476203d5bb9f1d in term 0.
I20260812 21:07:35.349817 9240 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c44605f34e84bb4930f84f51f063c6c" candidate_uuid: "3f2bca5a4eae4ae997476203d5bb9f1d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf288bd7fbf7423a8b982d5e57a2ed53" is_pre_election: true
I20260812 21:07:35.349997 9240 raft_consensus.cc:2454] T 4c44605f34e84bb4930f84f51f063c6c P cf288bd7fbf7423a8b982d5e57a2ed53 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f2bca5a4eae4ae997476203d5bb9f1d in term 0.
I20260812 21:07:35.350142 8924 leader_election.cc:302] T 4c44605f34e84bb4930f84f51f063c6c P 3f2bca5a4eae4ae997476203d5bb9f1d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 022a33213b32479face847c7df94069d, 3f2bca5a4eae4ae997476203d5bb9f1d; no voters:
I20260812 21:07:35.350289 9344 raft_consensus.cc:2781] T 4c44605f34e84bb4930f84f51f063c6c P 3f2bca5a4eae4ae997476203d5bb9f1d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 21:07:35.350353 9344 raft_consensus.cc:482] T 4c44605f34e84bb4930f84f51f063c6c P 3f2bca5a4eae4ae997476203d5bb9f1d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 21:07:35.350376 9344 raft_consensus.cc:3037] T 4c44605f34e84bb4930f84f51f063c6c P 3f2bca5a4eae4ae997476203d5bb9f1d [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:07:35.351078 9344 raft_consensus.cc:504] T 4c44605f34e84bb4930f84f51f063c6c P 3f2bca5a4eae4ae997476203d5bb9f1d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "022a33213b32479face847c7df94069d" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 46793 } } peers { permanent_uuid: "3f2bca5a4eae4ae997476203d5bb9f1d" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 35807 } } peers { permanent_uuid: "cf288bd7fbf7423a8b982d5e57a2ed53" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 40297 } }
I20260812 21:07:35.351208 9344 leader_election.cc:288] T 4c44605f34e84bb4930f84f51f063c6c P 3f2bca5a4eae4ae997476203d5bb9f1d [CANDIDATE]: Term 1 election: Requested vote from peers 022a33213b32479face847c7df94069d (127.11.254.129:46793), cf288bd7fbf7423a8b982d5e57a2ed53 (127.11.254.131:40297)
I20260812 21:07:35.351428 8797 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c44605f34e84bb4930f84f51f063c6c" candidate_uuid: "3f2bca5a4eae4ae997476203d5bb9f1d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "022a33213b32479face847c7df94069d"
I20260812 21:07:35.351420 9240 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c44605f34e84bb4930f84f51f063c6c" candidate_uuid: "3f2bca5a4eae4ae997476203d5bb9f1d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf288bd7fbf7423a8b982d5e57a2ed53"
I20260812 21:07:35.351522 9240 raft_consensus.cc:3037] T 4c44605f34e84bb4930f84f51f063c6c P cf288bd7fbf7423a8b982d5e57a2ed53 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:07:35.351523 8797 raft_consensus.cc:3037] T 4c44605f34e84bb4930f84f51f063c6c P 022a33213b32479face847c7df94069d [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:07:35.352425 9240 raft_consensus.cc:2454] T 4c44605f34e84bb4930f84f51f063c6c P cf288bd7fbf7423a8b982d5e57a2ed53 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f2bca5a4eae4ae997476203d5bb9f1d in term 1.
I20260812 21:07:35.352489 8797 raft_consensus.cc:2454] T 4c44605f34e84bb4930f84f51f063c6c P 022a33213b32479face847c7df94069d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f2bca5a4eae4ae997476203d5bb9f1d in term 1.
I20260812 21:07:35.352619 8925 leader_election.cc:302] T 4c44605f34e84bb4930f84f51f063c6c P 3f2bca5a4eae4ae997476203d5bb9f1d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f2bca5a4eae4ae997476203d5bb9f1d, cf288bd7fbf7423a8b982d5e57a2ed53; no voters:
I20260812 21:07:35.352715 9344 raft_consensus.cc:2781] T 4c44605f34e84bb4930f84f51f063c6c P 3f2bca5a4eae4ae997476203d5bb9f1d [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:07:35.352875 9344 raft_consensus.cc:686] T 4c44605f34e84bb4930f84f51f063c6c P 3f2bca5a4eae4ae997476203d5bb9f1d [term 1 LEADER]: Becoming Leader. State: Replica: 3f2bca5a4eae4ae997476203d5bb9f1d, State: Running, Role: LEADER
I20260812 21:07:35.352980 9344 consensus_queue.cc:237] T 4c44605f34e84bb4930f84f51f063c6c P 3f2bca5a4eae4ae997476203d5bb9f1d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "022a33213b32479face847c7df94069d" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 46793 } } peers { permanent_uuid: "3f2bca5a4eae4ae997476203d5bb9f1d" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 35807 } } peers { permanent_uuid: "cf288bd7fbf7423a8b982d5e57a2ed53" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 40297 } }
I20260812 21:07:35.353807 8572 catalog_manager.cc:5495] T 4c44605f34e84bb4930f84f51f063c6c P 3f2bca5a4eae4ae997476203d5bb9f1d reported cstate change: term changed from 0 to 1, leader changed from <none> to 3f2bca5a4eae4ae997476203d5bb9f1d (127.11.254.130). New cstate: current_term: 1 leader_uuid: "3f2bca5a4eae4ae997476203d5bb9f1d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "022a33213b32479face847c7df94069d" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 46793 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3f2bca5a4eae4ae997476203d5bb9f1d" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 35807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cf288bd7fbf7423a8b982d5e57a2ed53" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 40297 } health_report { overall_health: UNKNOWN } } }
I20260812 21:07:35.394367 9333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:35.391976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49452 (local address 127.11.254.130:35807)
0812 21:07:35.392044 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0812 21:07:35.392046 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:07:35.392054 (+ 8us) server_negotiation.cc:407] Connection header received
0812 21:07:35.392103 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:07:35.392104 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:07:35.392125 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:07:35.392185 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 21:07:35.392307 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:35.392850 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:35.393592 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:35.393786 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:35.393838 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:07:35.394038 (+ 200us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 21:07:35.394145 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 21:07:35.394218 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 21:07:35.394240 (+ 22us) server_negotiation.cc:299] Negotiation successful
0812 21:07:35.394263 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260812 21:07:35.396315 8797 raft_consensus.cc:1261] T 4c44605f34e84bb4930f84f51f063c6c P 022a33213b32479face847c7df94069d [term 1 FOLLOWER]: Refusing update from remote peer 3f2bca5a4eae4ae997476203d5bb9f1d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 21:07:35.396314 9240 raft_consensus.cc:1261] T 4c44605f34e84bb4930f84f51f063c6c P cf288bd7fbf7423a8b982d5e57a2ed53 [term 1 FOLLOWER]: Refusing update from remote peer 3f2bca5a4eae4ae997476203d5bb9f1d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 21:07:35.396670 9344 consensus_queue.cc:1035] T 4c44605f34e84bb4930f84f51f063c6c P 3f2bca5a4eae4ae997476203d5bb9f1d [LEADER]: Connected to new peer: Peer: permanent_uuid: "022a33213b32479face847c7df94069d" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 46793 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 21:07:35.396873 9350 consensus_queue.cc:1035] T 4c44605f34e84bb4930f84f51f063c6c P 3f2bca5a4eae4ae997476203d5bb9f1d [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf288bd7fbf7423a8b982d5e57a2ed53" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 40297 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 21:07:35.399314 9354 mvcc.cc:205] Tried to move back new op lower bound from 7317786031700549632 to 7317786031526297600. Current Snapshot: MvccSnapshot[applied={T|T < 7317786031700549632}]
I20260812 21:07:35.399430 9357 mvcc.cc:205] Tried to move back new op lower bound from 7317786031700549632 to 7317786031526297600. Current Snapshot: MvccSnapshot[applied={T|T < 7317786031700549632}]
I20260812 21:07:35.402338 8572 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50416:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 21:07:35.402448 8572 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50416:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 21:07:35.403270 8572 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ff7f435573604945a810f2497612a6e7: Sending DeleteTablet for 3 replicas of tablet 4c44605f34e84bb4930f84f51f063c6c
I20260812 21:07:35.403656 8769 tablet_service.cc:1514] Processing DeleteTablet for tablet 4c44605f34e84bb4930f84f51f063c6c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:07:35 UTC) from {username='kudu', principal='kudu/127.11.254.190@KRBTEST.COM'} at 127.0.0.1:47362
I20260812 21:07:35.403728 8994 tablet_service.cc:1514] Processing DeleteTablet for tablet 4c44605f34e84bb4930f84f51f063c6c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:07:35 UTC) from {username='kudu', principal='kudu/127.11.254.190@KRBTEST.COM'} at 127.0.0.1:49440
I20260812 21:07:35.403930 9366 tablet_replica.cc:330] T 4c44605f34e84bb4930f84f51f063c6c P 022a33213b32479face847c7df94069d: stopping tablet replica
I20260812 21:07:35.404078 9366 raft_consensus.cc:2229] T 4c44605f34e84bb4930f84f51f063c6c P 022a33213b32479face847c7df94069d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 21:07:35.404098 9201 tablet_service.cc:1514] Processing DeleteTablet for tablet 4c44605f34e84bb4930f84f51f063c6c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:07:35 UTC) from {username='kudu', principal='kudu/127.11.254.190@KRBTEST.COM'} at 127.0.0.1:50380
I20260812 21:07:35.404116 9367 tablet_replica.cc:330] T 4c44605f34e84bb4930f84f51f063c6c P 3f2bca5a4eae4ae997476203d5bb9f1d: stopping tablet replica
I20260812 21:07:35.404237 9367 raft_consensus.cc:2229] T 4c44605f34e84bb4930f84f51f063c6c P 3f2bca5a4eae4ae997476203d5bb9f1d [term 1 LEADER]: Raft consensus shutting down.
I20260812 21:07:35.404246 9366 raft_consensus.cc:2258] T 4c44605f34e84bb4930f84f51f063c6c P 022a33213b32479face847c7df94069d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 21:07:35.404475 9367 raft_consensus.cc:2258] T 4c44605f34e84bb4930f84f51f063c6c P 3f2bca5a4eae4ae997476203d5bb9f1d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 21:07:35.404630 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 8665
I20260812 21:07:35.404719 9366 ts_tablet_manager.cc:1905] T 4c44605f34e84bb4930f84f51f063c6c P 022a33213b32479face847c7df94069d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 21:07:35.409642 9367 ts_tablet_manager.cc:1905] T 4c44605f34e84bb4930f84f51f063c6c P 3f2bca5a4eae4ae997476203d5bb9f1d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 21:07:35.411929 9367 ts_tablet_manager.cc:1918] T 4c44605f34e84bb4930f84f51f063c6c P 3f2bca5a4eae4ae997476203d5bb9f1d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 21:07:35.412034 9367 log.cc:1196] T 4c44605f34e84bb4930f84f51f063c6c P 3f2bca5a4eae4ae997476203d5bb9f1d: Deleting WAL directory at /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568607354921-12282-0/minicluster-data/ts-1/wal/wals/4c44605f34e84bb4930f84f51f063c6c
I20260812 21:07:35.412405 9367 ts_tablet_manager.cc:1939] T 4c44605f34e84bb4930f84f51f063c6c P 3f2bca5a4eae4ae997476203d5bb9f1d: Deleting consensus metadata
I20260812 21:07:35.413018 8544 catalog_manager.cc:4841] TS 3f2bca5a4eae4ae997476203d5bb9f1d (127.11.254.130:35807): tablet 4c44605f34e84bb4930f84f51f063c6c (table test-table [id=7c2cb47840664ba0ba77d2d7089f2b58]) successfully deleted
I20260812 21:07:35.418051 9368 tablet_replica.cc:330] T 4c44605f34e84bb4930f84f51f063c6c P cf288bd7fbf7423a8b982d5e57a2ed53: stopping tablet replica
I20260812 21:07:35.418231 9368 raft_consensus.cc:2229] T 4c44605f34e84bb4930f84f51f063c6c P cf288bd7fbf7423a8b982d5e57a2ed53 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 21:07:35.418370 9368 raft_consensus.cc:2258] T 4c44605f34e84bb4930f84f51f063c6c P cf288bd7fbf7423a8b982d5e57a2ed53 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 21:07:35.418919 9368 ts_tablet_manager.cc:1905] T 4c44605f34e84bb4930f84f51f063c6c P cf288bd7fbf7423a8b982d5e57a2ed53: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 21:07:35.420742 9368 ts_tablet_manager.cc:1918] T 4c44605f34e84bb4930f84f51f063c6c P cf288bd7fbf7423a8b982d5e57a2ed53: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 21:07:35.420833 9368 log.cc:1196] T 4c44605f34e84bb4930f84f51f063c6c P cf288bd7fbf7423a8b982d5e57a2ed53: Deleting WAL directory at /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786568607354921-12282-0/minicluster-data/ts-2/wal/wals/4c44605f34e84bb4930f84f51f063c6c
I20260812 21:07:35.421105 9368 ts_tablet_manager.cc:1939] T 4c44605f34e84bb4930f84f51f063c6c P cf288bd7fbf7423a8b982d5e57a2ed53: Deleting consensus metadata
I20260812 21:07:35.421645 8550 catalog_manager.cc:4841] TS cf288bd7fbf7423a8b982d5e57a2ed53 (127.11.254.131:40297): tablet 4c44605f34e84bb4930f84f51f063c6c (table test-table [id=7c2cb47840664ba0ba77d2d7089f2b58]) successfully deleted
W20260812 21:07:35.422171 8549 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.254.129:46793 (error 108)
I20260812 21:07:35.422576 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 8892
I20260812 21:07:35.428802 9329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:35.428601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.129:46793 (local address 127.0.0.1:47376)
0812 21:07:35.428677 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0812 21:07:35.428751 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.129:46793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260812 21:07:35.430450 8549 catalog_manager.cc:4584] TS 022a33213b32479face847c7df94069d (127.11.254.129:46793): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4c44605f34e84bb4930f84f51f063c6c: Network error: Client connection negotiation failed: client connection to 127.11.254.129:46793: connect: Connection refused (error 111)
I20260812 21:07:35.436333 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 9103
I20260812 21:07:35.448674 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 8519
2026-08-12T21:07:35Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2340 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-12T21:07:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T21:07:35Z Disabled control of system clock
I20260812 21:07:35.487380 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786568607354921-12282-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786568607354921-12282-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786568607354921-12282-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.254.190:32907
--webserver_interface=127.11.254.190
--webserver_port=0
--builtin_ntp_servers=127.11.254.148:42891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.254.190:32907
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 21:07:35.571218 9374 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:07:35.571465 9374 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:07:35.571485 9374 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:07:35.572647 9374 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 21:07:35.572693 9374 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:07:35.572705 9374 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:07:35.572716 9374 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 21:07:35.572727 9374 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 21:07:35.574081 9374 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.254.148:42891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786568607354921-12282-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786568607354921-12282-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.254.190:32907
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.254.190:32907
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.254.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9374
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786568607354921-12282-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:07:35.574291 9374 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:07:35.574551 9374 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:07:35.579521 9388 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:07:35.579526 9389 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:07:35.579865 9393 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:07:35.580077 9374 server_base.cc:1034] running on GCE node
I20260812 21:07:35.580287 9374 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:07:35.580580 9374 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:07:35.581749 9374 hybrid_clock.cc:648] HybridClock initialized: now 1786568855581729 us; error 35 us; skew 500 ppm
I20260812 21:07:35.583150 9374 webserver.cc:460] Webserver started at http://127.11.254.190:46523/ using document root <none> and password file <none>
I20260812 21:07:35.583354 9374 fs_manager.cc:359] Metadata directory not provided
I20260812 21:07:35.583395 9374 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:07:35.583488 9374 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:07:35.584579 9374 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786568607354921-12282-0/minicluster-data/master-0/data/instance:
uuid: "d3f0ec64844d40d98e0bd0ba697e774d"
format_stamp: "Formatted at 2026-08-12 21:07:35 on dist-test-slave-vpvm"
server_key: "636987a4f554db4921cb5ca5d0fd6520"
server_key_iv: "68309841c8f7cd0994b7107eb7e38767"
server_key_version: "encryptionkey@0"
I20260812 21:07:35.584892 9374 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786568607354921-12282-0/minicluster-data/master-0/wal/instance:
uuid: "d3f0ec64844d40d98e0bd0ba697e774d"
format_stamp: "Formatted at 2026-08-12 21:07:35 on dist-test-slave-vpvm"
server_key: "636987a4f554db4921cb5ca5d0fd6520"
server_key_iv: "68309841c8f7cd0994b7107eb7e38767"
server_key_version: "encryptionkey@0"
I20260812 21:07:35.586180 9374 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 21:07:35.586949 9404 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:07:35.587106 9374 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 21:07:35.587193 9374 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786568607354921-12282-0/minicluster-data/master-0/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786568607354921-12282-0/minicluster-data/master-0/wal
uuid: "d3f0ec64844d40d98e0bd0ba697e774d"
format_stamp: "Formatted at 2026-08-12 21:07:35 on dist-test-slave-vpvm"
server_key: "636987a4f554db4921cb5ca5d0fd6520"
server_key_iv: "68309841c8f7cd0994b7107eb7e38767"
server_key_version: "encryptionkey@0"
I20260812 21:07:35.587255 9374 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786568607354921-12282-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786568607354921-12282-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786568607354921-12282-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:07:35.604334 9374 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:07:35.604692 9374 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:07:35.604835 9374 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:07:35.608970 9374 rpc_server.cc:307] RPC server started. Bound to: 127.11.254.190:32907
I20260812 21:07:35.609010 9487 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.254.190:32907 every 8 connection(s)
I20260812 21:07:35.609342 9374 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
I20260812 21:07:35.610359 9488 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:07:35.612354 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 9374
I20260812 21:07:35.612442 12282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786568607354921-12282-0/minicluster-data/master-0/wal/instance
I20260812 21:07:35.612624 12282 external_mini_cluster.cc:1428] Setting key 4943ad8edf7ef1630be1768ffad74f0a
I20260812 21:07:35.612736 9488 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d3f0ec64844d40d98e0bd0ba697e774d: Bootstrap starting.
I20260812 21:07:35.613415 9488 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d3f0ec64844d40d98e0bd0ba697e774d: Neither blocks nor log segments found. Creating new log.
I20260812 21:07:35.613699 9488 log.cc:824] T 00000000000000000000000000000000 P d3f0ec64844d40d98e0bd0ba697e774d: Log is configured to *not* fsync() on all Append() calls
I20260812 21:07:35.614485 9488 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d3f0ec64844d40d98e0bd0ba697e774d: No bootstrap required, opened a new log
I20260812 21:07:35.616569 9488 raft_consensus.cc:348] T 00000000000000000000000000000000 P d3f0ec64844d40d98e0bd0ba697e774d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3f0ec64844d40d98e0bd0ba697e774d" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 32907 } }
I20260812 21:07:35.616722 9488 raft_consensus.cc:374] T 00000000000000000000000000000000 P d3f0ec64844d40d98e0bd0ba697e774d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:07:35.616758 9488 raft_consensus.cc:729] T 00000000000000000000000000000000 P d3f0ec64844d40d98e0bd0ba697e774d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3f0ec64844d40d98e0bd0ba697e774d, State: Initialized, Role: FOLLOWER
I20260812 21:07:35.616940 9488 consensus_queue.cc:260] T 00000000000000000000000000000000 P d3f0ec64844d40d98e0bd0ba697e774d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3f0ec64844d40d98e0bd0ba697e774d" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 32907 } }
I20260812 21:07:35.617033 9488 raft_consensus.cc:388] T 00000000000000000000000000000000 P d3f0ec64844d40d98e0bd0ba697e774d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 21:07:35.617063 9488 raft_consensus.cc:482] T 00000000000000000000000000000000 P d3f0ec64844d40d98e0bd0ba697e774d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 21:07:35.617113 9488 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d3f0ec64844d40d98e0bd0ba697e774d [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:07:35.617779 9488 raft_consensus.cc:504] T 00000000000000000000000000000000 P d3f0ec64844d40d98e0bd0ba697e774d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3f0ec64844d40d98e0bd0ba697e774d" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 32907 } }
I20260812 21:07:35.617902 9488 leader_election.cc:302] T 00000000000000000000000000000000 P d3f0ec64844d40d98e0bd0ba697e774d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d3f0ec64844d40d98e0bd0ba697e774d; no voters:
I20260812 21:07:35.618124 9488 leader_election.cc:288] T 00000000000000000000000000000000 P d3f0ec64844d40d98e0bd0ba697e774d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 21:07:35.618166 9493 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d3f0ec64844d40d98e0bd0ba697e774d [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:07:35.618188 9491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:35.613139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55506 (local address 127.11.254.190:32907)
0812 21:07:35.613324 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0812 21:07:35.613331 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 21:07:35.613355 (+ 24us) server_negotiation.cc:407] Connection header received
0812 21:07:35.613801 (+ 446us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:07:35.613810 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:07:35.613876 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:07:35.614052 (+ 176us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:07:35.614221 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:35.615171 (+ 950us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:35.616237 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:35.616644 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:35.616736 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:07:35.617452 (+ 716us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:07:35.617455 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:07:35.617456 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 21:07:35.617658 (+ 202us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:07:35.617693 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 21:07:35.617704 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:07:35.617753 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0812 21:07:35.617866 (+ 113us) server_negotiation.cc:299] Negotiation successful
0812 21:07:35.617984 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":65,"threads_started":1}
I20260812 21:07:35.618326 9493 raft_consensus.cc:686] T 00000000000000000000000000000000 P d3f0ec64844d40d98e0bd0ba697e774d [term 1 LEADER]: Becoming Leader. State: Replica: d3f0ec64844d40d98e0bd0ba697e774d, State: Running, Role: LEADER
I20260812 21:07:35.618403 9488 sys_catalog.cc:564] T 00000000000000000000000000000000 P d3f0ec64844d40d98e0bd0ba697e774d [sys.catalog]: configured and running, proceeding with master startup.
I20260812 21:07:35.618440 9493 consensus_queue.cc:237] T 00000000000000000000000000000000 P d3f0ec64844d40d98e0bd0ba697e774d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3f0ec64844d40d98e0bd0ba697e774d" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 32907 } }
I20260812 21:07:35.618773 9493 sys_catalog.cc:455] T 00000000000000000000000000000000 P d3f0ec64844d40d98e0bd0ba697e774d [sys.catalog]: SysCatalogTable state changed. Reason: New leader d3f0ec64844d40d98e0bd0ba697e774d. Latest consensus state: current_term: 1 leader_uuid: "d3f0ec64844d40d98e0bd0ba697e774d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3f0ec64844d40d98e0bd0ba697e774d" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 32907 } } }
I20260812 21:07:35.618860 9495 sys_catalog.cc:455] T 00000000000000000000000000000000 P d3f0ec64844d40d98e0bd0ba697e774d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d3f0ec64844d40d98e0bd0ba697e774d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3f0ec64844d40d98e0bd0ba697e774d" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 32907 } } }
I20260812 21:07:35.618927 9493 sys_catalog.cc:458] T 00000000000000000000000000000000 P d3f0ec64844d40d98e0bd0ba697e774d [sys.catalog]: This master's current role is: LEADER
I20260812 21:07:35.619091 9495 sys_catalog.cc:458] T 00000000000000000000000000000000 P d3f0ec64844d40d98e0bd0ba697e774d [sys.catalog]: This master's current role is: LEADER
I20260812 21:07:35.619280 9510 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 21:07:35.619794 9510 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 21:07:35.621273 9510 catalog_manager.cc:1347] Generated new cluster ID: 367192d1733142c09772e08f80c97b25
I20260812 21:07:35.621318 9510 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 21:07:35.627679 9510 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 21:07:35.628201 9510 catalog_manager.cc:1504] Loading token signing keys...
I20260812 21:07:35.635528 9510 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d3f0ec64844d40d98e0bd0ba697e774d: Generated new TSK 0
I20260812 21:07:35.635735 9510 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 21:07:35.638289 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786568607354921-12282-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786568607354921-12282-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786568607354921-12282-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786568607354921-12282-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.254.129:0
--local_ip_for_outbound_sockets=127.11.254.129
--webserver_interface=127.11.254.129
--webserver_port=0
--tserver_master_addrs=127.11.254.190:32907
--builtin_ntp_servers=127.11.254.148:42891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 21:07:35.743546 9523 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:07:35.743764 9523 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:07:35.743793 9523 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:07:35.745005 9523 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:07:35.745090 9523 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:07:35.745146 9523 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.254.129
I20260812 21:07:35.746695 9523 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.254.148:42891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786568607354921-12282-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786568607354921-12282-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.254.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786568607354921-12282-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.254.129
--webserver_port=0
--tserver_master_addrs=127.11.254.190:32907
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9523
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.254.129
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786568607354921-12282-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:07:35.746969 9523 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:07:35.747269 9523 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:07:35.753455 9529 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:07:35.753580 9532 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:07:35.753671 9523 server_base.cc:1034] running on GCE node
W20260812 21:07:35.753723 9534 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:07:35.753986 9523 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:07:35.754253 9523 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:07:35.755426 9523 hybrid_clock.cc:648] HybridClock initialized: now 1786568855755409 us; error 29 us; skew 500 ppm
I20260812 21:07:35.756711 9523 webserver.cc:460] Webserver started at http://127.11.254.129:43541/ using document root <none> and password file <none>
I20260812 21:07:35.756947 9523 fs_manager.cc:359] Metadata directory not provided
I20260812 21:07:35.756997 9523 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:07:35.757129 9523 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:07:35.758262 9523 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786568607354921-12282-0/minicluster-data/ts-0/data/instance:
uuid: "16b5a2eec7814f65bc01fd3526747fc1"
format_stamp: "Formatted at 2026-08-12 21:07:35 on dist-test-slave-vpvm"
server_key: "e22028d680d09788368c44fd00432334"
server_key_iv: "326fcafc1c365e57683cb950f0c92fe9"
server_key_version: "encryptionkey@0"
I20260812 21:07:35.758605 9523 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786568607354921-12282-0/minicluster-data/ts-0/wal/instance:
uuid: "16b5a2eec7814f65bc01fd3526747fc1"
format_stamp: "Formatted at 2026-08-12 21:07:35 on dist-test-slave-vpvm"
server_key: "e22028d680d09788368c44fd00432334"
server_key_iv: "326fcafc1c365e57683cb950f0c92fe9"
server_key_version: "encryptionkey@0"
I20260812 21:07:35.759871 9523 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 21:07:35.760602 9541 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:07:35.760797 9523 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 21:07:35.760902 9523 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786568607354921-12282-0/minicluster-data/ts-0/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786568607354921-12282-0/minicluster-data/ts-0/wal
uuid: "16b5a2eec7814f65bc01fd3526747fc1"
format_stamp: "Formatted at 2026-08-12 21:07:35 on dist-test-slave-vpvm"
server_key: "e22028d680d09788368c44fd00432334"
server_key_iv: "326fcafc1c365e57683cb950f0c92fe9"
server_key_version: "encryptionkey@0"
I20260812 21:07:35.760978 9523 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786568607354921-12282-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786568607354921-12282-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786568607354921-12282-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:07:35.773103 9523 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:07:35.773471 9523 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:07:35.773614 9523 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:07:35.773856 9523 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:07:35.774282 9523 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:07:35.774333 9523 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:07:35.774360 9523 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:07:35.774384 9523 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:07:35.780434 9523 rpc_server.cc:307] RPC server started. Bound to: 127.11.254.129:44791
I20260812 21:07:35.780473 9765 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.254.129:44791 every 8 connection(s)
I20260812 21:07:35.780834 9523 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786568607354921-12282-0/minicluster-data/ts-0/data/info.pb
I20260812 21:07:35.785012 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 9523
I20260812 21:07:35.785109 12282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786568607354921-12282-0/minicluster-data/ts-0/wal/instance
I20260812 21:07:35.785293 12282 external_mini_cluster.cc:1428] Setting key c80a02fcaafabda21ca66ed72a69091e
I20260812 21:07:35.786242 9491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:35.781602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.129:43439 (local address 127.11.254.190:32907)
0812 21:07:35.781661 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0812 21:07:35.781663 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:07:35.781849 (+ 186us) server_negotiation.cc:407] Connection header received
0812 21:07:35.782412 (+ 563us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:07:35.782414 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:07:35.782434 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:07:35.782486 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:07:35.783187 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:35.783735 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:35.784729 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:35.784917 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:35.784975 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:07:35.785851 (+ 876us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:07:35.785852 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:07:35.785852 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 21:07:35.785856 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:07:35.785874 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 21:07:35.785879 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:07:35.785927 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0812 21:07:35.786135 (+ 208us) server_negotiation.cc:299] Negotiation successful
0812 21:07:35.786163 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260812 21:07:35.786273 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786568607354921-12282-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786568607354921-12282-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786568607354921-12282-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786568607354921-12282-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.254.130:0
--local_ip_for_outbound_sockets=127.11.254.130
--webserver_interface=127.11.254.130
--webserver_port=0
--tserver_master_addrs=127.11.254.190:32907
--builtin_ntp_servers=127.11.254.148:42891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 21:07:35.786479 9768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:35.781516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:32907 (local address 127.11.254.129:43439)
0812 21:07:35.781731 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0812 21:07:35.781767 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0812 21:07:35.782312 (+ 545us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:07:35.782568 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:07:35.782572 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:07:35.782583 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:07:35.783087 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:07:35.783092 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:07:35.783825 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:07:35.783826 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:07:35.784645 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:07:35.784647 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:07:35.784987 (+ 340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:07:35.784988 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:07:35.785124 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:07:35.785699 (+ 575us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 21:07:35.785700 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:07:35.785738 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 21:07:35.785766 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:07:35.786052 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:07:35.786058 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:07:35.786061 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 21:07:35.786127 (+ 66us) client_negotiation.cc:240] Negotiation successful
0812 21:07:35.786236 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":97,"threads_started":1}
I20260812 21:07:35.786938 9766 heartbeater.cc:347] Connected to a master server at 127.11.254.190:32907
I20260812 21:07:35.787026 9766 heartbeater.cc:464] Registering TS with master...
I20260812 21:07:35.787214 9766 heartbeater.cc:511] Master 127.11.254.190:32907 requested a full tablet report, sending...
I20260812 21:07:35.787719 9433 ts_manager.cc:194] Registered new tserver with Master: 16b5a2eec7814f65bc01fd3526747fc1 (127.11.254.129:44791)
I20260812 21:07:35.788362 9433 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.254.129:43439
W20260812 21:07:35.869590 9772 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:07:35.869807 9772 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:07:35.869828 9772 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:07:35.871004 9772 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:07:35.871044 9772 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:07:35.871069 9772 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.254.130
I20260812 21:07:35.872371 9772 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.254.148:42891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786568607354921-12282-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786568607354921-12282-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.254.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786568607354921-12282-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.254.130
--webserver_port=0
--tserver_master_addrs=127.11.254.190:32907
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9772
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.254.130
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786568607354921-12282-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:07:35.872571 9772 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:07:35.872831 9772 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:07:35.877818 9782 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:07:35.877821 9786 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:07:35.877821 9781 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:07:35.877943 9772 server_base.cc:1034] running on GCE node
I20260812 21:07:35.878268 9772 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:07:35.878545 9772 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:07:35.879715 9772 hybrid_clock.cc:648] HybridClock initialized: now 1786568855879699 us; error 28 us; skew 500 ppm
I20260812 21:07:35.880991 9772 webserver.cc:460] Webserver started at http://127.11.254.130:46213/ using document root <none> and password file <none>
I20260812 21:07:35.881220 9772 fs_manager.cc:359] Metadata directory not provided
I20260812 21:07:35.881270 9772 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:07:35.881402 9772 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:07:35.882619 9772 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786568607354921-12282-0/minicluster-data/ts-1/data/instance:
uuid: "72ce0f1234f14bccb39dd34d942fc51c"
format_stamp: "Formatted at 2026-08-12 21:07:35 on dist-test-slave-vpvm"
server_key: "6fa21c654aff2274f715e0553d8a2d87"
server_key_iv: "7c14e4dcdbde89c001f7aeeec3588c5d"
server_key_version: "encryptionkey@0"
I20260812 21:07:35.882982 9772 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786568607354921-12282-0/minicluster-data/ts-1/wal/instance:
uuid: "72ce0f1234f14bccb39dd34d942fc51c"
format_stamp: "Formatted at 2026-08-12 21:07:35 on dist-test-slave-vpvm"
server_key: "6fa21c654aff2274f715e0553d8a2d87"
server_key_iv: "7c14e4dcdbde89c001f7aeeec3588c5d"
server_key_version: "encryptionkey@0"
I20260812 21:07:35.884323 9772 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260812 21:07:35.885078 9792 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:07:35.885249 9772 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 21:07:35.885335 9772 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786568607354921-12282-0/minicluster-data/ts-1/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786568607354921-12282-0/minicluster-data/ts-1/wal
uuid: "72ce0f1234f14bccb39dd34d942fc51c"
format_stamp: "Formatted at 2026-08-12 21:07:35 on dist-test-slave-vpvm"
server_key: "6fa21c654aff2274f715e0553d8a2d87"
server_key_iv: "7c14e4dcdbde89c001f7aeeec3588c5d"
server_key_version: "encryptionkey@0"
I20260812 21:07:35.885407 9772 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786568607354921-12282-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786568607354921-12282-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786568607354921-12282-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:07:35.927773 9772 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:07:35.928155 9772 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:07:35.928304 9772 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:07:35.928556 9772 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:07:35.928901 9772 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:07:35.928944 9772 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:07:35.928975 9772 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:07:35.928993 9772 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:07:35.934922 9772 rpc_server.cc:307] RPC server started. Bound to: 127.11.254.130:45765
I20260812 21:07:35.934960 9985 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.254.130:45765 every 8 connection(s)
I20260812 21:07:35.935348 9772 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786568607354921-12282-0/minicluster-data/ts-1/data/info.pb
I20260812 21:07:35.940855 9491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:35.936181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.130:46119 (local address 127.11.254.190:32907)
0812 21:07:35.936270 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0812 21:07:35.936273 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 21:07:35.936472 (+ 199us) server_negotiation.cc:407] Connection header received
0812 21:07:35.937006 (+ 534us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:07:35.937008 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:07:35.937019 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:07:35.937084 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:07:35.937782 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:35.938339 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:35.939414 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:35.939663 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:35.939730 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:07:35.940604 (+ 874us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:07:35.940606 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:07:35.940606 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 21:07:35.940611 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:07:35.940628 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 21:07:35.940631 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:07:35.940682 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0812 21:07:35.940758 (+ 76us) server_negotiation.cc:299] Negotiation successful
0812 21:07:35.940789 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260812 21:07:35.941073 9989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:35.936152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:32907 (local address 127.11.254.130:46119)
0812 21:07:35.936344 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0812 21:07:35.936383 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 21:07:35.936902 (+ 519us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:07:35.937150 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:07:35.937154 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:07:35.937163 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:07:35.937700 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:07:35.937705 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:07:35.938436 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:07:35.938437 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:07:35.939290 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:07:35.939293 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:07:35.939753 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:07:35.939753 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:07:35.939883 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:07:35.940445 (+ 562us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 21:07:35.940445 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:07:35.940476 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 21:07:35.940502 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:07:35.940694 (+ 192us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:07:35.940698 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:07:35.940701 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 21:07:35.940753 (+ 52us) client_negotiation.cc:240] Negotiation successful
0812 21:07:35.940845 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":85,"threads_started":1}
I20260812 21:07:35.941565 9986 heartbeater.cc:347] Connected to a master server at 127.11.254.190:32907
I20260812 21:07:35.941670 9986 heartbeater.cc:464] Registering TS with master...
I20260812 21:07:35.941860 9986 heartbeater.cc:511] Master 127.11.254.190:32907 requested a full tablet report, sending...
I20260812 21:07:35.942277 9433 ts_manager.cc:194] Registered new tserver with Master: 72ce0f1234f14bccb39dd34d942fc51c (127.11.254.130:45765)
I20260812 21:07:35.942672 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 9772
I20260812 21:07:35.942776 12282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786568607354921-12282-0/minicluster-data/ts-1/wal/instance
I20260812 21:07:35.942819 9433 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.254.130:46119
I20260812 21:07:35.943001 12282 external_mini_cluster.cc:1428] Setting key 4588364f60d5085edd3fca7f17a007ad
I20260812 21:07:35.943940 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786568607354921-12282-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786568607354921-12282-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786568607354921-12282-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786568607354921-12282-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.254.131:0
--local_ip_for_outbound_sockets=127.11.254.131
--webserver_interface=127.11.254.131
--webserver_port=0
--tserver_master_addrs=127.11.254.190:32907
--builtin_ntp_servers=127.11.254.148:42891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 21:07:36.037022 9990 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:07:36.037225 9990 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:07:36.037249 9990 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:07:36.038494 9990 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:07:36.038537 9990 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:07:36.038563 9990 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.254.131
I20260812 21:07:36.040037 9990 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.254.148:42891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786568607354921-12282-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786568607354921-12282-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.254.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786568607354921-12282-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.254.131
--webserver_port=0
--tserver_master_addrs=127.11.254.190:32907
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9990
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.254.131
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786568607354921-12282-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:07:36.040278 9990 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:07:36.040580 9990 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:07:36.046389 9999 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:07:36.046411 9996 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:07:36.046710 10002 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:07:36.049355 9990 server_base.cc:1034] running on GCE node
I20260812 21:07:36.049588 9990 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:07:36.049826 9990 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:07:36.050983 9990 hybrid_clock.cc:648] HybridClock initialized: now 1786568856050978 us; error 59 us; skew 500 ppm
I20260812 21:07:36.052302 9990 webserver.cc:460] Webserver started at http://127.11.254.131:42789/ using document root <none> and password file <none>
I20260812 21:07:36.052614 9990 fs_manager.cc:359] Metadata directory not provided
I20260812 21:07:36.052665 9990 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:07:36.052783 9990 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:07:36.053925 9990 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786568607354921-12282-0/minicluster-data/ts-2/data/instance:
uuid: "e598baf6017a4899a58a037fd0afca4d"
format_stamp: "Formatted at 2026-08-12 21:07:36 on dist-test-slave-vpvm"
server_key: "a18a0963ed5940fd6715c77465be12d7"
server_key_iv: "d8bc2fe0996b666cacbdcc5dc5abd9ec"
server_key_version: "encryptionkey@0"
I20260812 21:07:36.054349 9990 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786568607354921-12282-0/minicluster-data/ts-2/wal/instance:
uuid: "e598baf6017a4899a58a037fd0afca4d"
format_stamp: "Formatted at 2026-08-12 21:07:36 on dist-test-slave-vpvm"
server_key: "a18a0963ed5940fd6715c77465be12d7"
server_key_iv: "d8bc2fe0996b666cacbdcc5dc5abd9ec"
server_key_version: "encryptionkey@0"
I20260812 21:07:36.055791 9990 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 21:07:36.056511 10011 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:07:36.056653 9990 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 21:07:36.056742 9990 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786568607354921-12282-0/minicluster-data/ts-2/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786568607354921-12282-0/minicluster-data/ts-2/wal
uuid: "e598baf6017a4899a58a037fd0afca4d"
format_stamp: "Formatted at 2026-08-12 21:07:36 on dist-test-slave-vpvm"
server_key: "a18a0963ed5940fd6715c77465be12d7"
server_key_iv: "d8bc2fe0996b666cacbdcc5dc5abd9ec"
server_key_version: "encryptionkey@0"
I20260812 21:07:36.056816 9990 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786568607354921-12282-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786568607354921-12282-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786568607354921-12282-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:07:36.077054 9990 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:07:36.077380 9990 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:07:36.077524 9990 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:07:36.077775 9990 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:07:36.078178 9990 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:07:36.078215 9990 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:07:36.078246 9990 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:07:36.078270 9990 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:07:36.084043 9990 rpc_server.cc:307] RPC server started. Bound to: 127.11.254.131:40515
I20260812 21:07:36.084100 10197 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.254.131:40515 every 8 connection(s)
I20260812 21:07:36.084481 9990 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786568607354921-12282-0/minicluster-data/ts-2/data/info.pb
I20260812 21:07:36.089998 9491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:36.085267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.131:43813 (local address 127.11.254.190:32907)
0812 21:07:36.085330 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0812 21:07:36.085333 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 21:07:36.085577 (+ 244us) server_negotiation.cc:407] Connection header received
0812 21:07:36.086150 (+ 573us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:07:36.086152 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:07:36.086165 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:07:36.086212 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:07:36.086997 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:36.087503 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:36.088527 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:36.088744 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:36.088801 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:07:36.089672 (+ 871us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:07:36.089673 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:07:36.089674 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 21:07:36.089679 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:07:36.089695 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 21:07:36.089699 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:07:36.089753 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0812 21:07:36.089841 (+ 88us) server_negotiation.cc:299] Negotiation successful
0812 21:07:36.089868 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260812 21:07:36.090034 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 9990
I20260812 21:07:36.090162 12282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786568607354921-12282-0/minicluster-data/ts-2/wal/instance
I20260812 21:07:36.090188 10201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:36.085214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:32907 (local address 127.11.254.131:43813)
0812 21:07:36.085436 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0812 21:07:36.085475 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 21:07:36.086048 (+ 573us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:07:36.086293 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:07:36.086296 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:07:36.086305 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:07:36.086904 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:07:36.086911 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:07:36.087590 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:07:36.087590 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:07:36.088445 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:07:36.088448 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:07:36.088816 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:07:36.088817 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:07:36.088941 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:07:36.089484 (+ 543us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 21:07:36.089485 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:07:36.089565 (+ 80us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 21:07:36.089589 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:07:36.089752 (+ 163us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:07:36.089758 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:07:36.089761 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 21:07:36.089821 (+ 60us) client_negotiation.cc:240] Negotiation successful
0812 21:07:36.089929 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":92,"threads_started":1}
I20260812 21:07:36.090415 12282 external_mini_cluster.cc:1428] Setting key 8ba02349c7736ad74d3fed5e4f9438fd
I20260812 21:07:36.090762 10198 heartbeater.cc:347] Connected to a master server at 127.11.254.190:32907
I20260812 21:07:36.090862 10198 heartbeater.cc:464] Registering TS with master...
I20260812 21:07:36.091055 10198 heartbeater.cc:511] Master 127.11.254.190:32907 requested a full tablet report, sending...
I20260812 21:07:36.091410 9433 ts_manager.cc:194] Registered new tserver with Master: e598baf6017a4899a58a037fd0afca4d (127.11.254.131:40515)
I20260812 21:07:36.091805 9433 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.254.131:43813
I20260812 21:07:36.102437 12282 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 21:07:36.106894 9491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:36.103948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55512 (local address 127.11.254.190:32907)
0812 21:07:36.104013 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0812 21:07:36.104015 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:07:36.104113 (+ 98us) server_negotiation.cc:407] Connection header received
0812 21:07:36.104175 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:07:36.104176 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:07:36.104187 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:07:36.104241 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:07:36.104392 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:36.104883 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:36.105681 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:36.105911 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:36.105986 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:07:36.106661 (+ 675us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:07:36.106663 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:07:36.106663 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 21:07:36.106668 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:07:36.106684 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 21:07:36.106688 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:07:36.106733 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0812 21:07:36.106785 (+ 52us) server_negotiation.cc:299] Negotiation successful
0812 21:07:36.106809 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260812 21:07:36.108328 9433 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55512:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 21:07:36.109831 9433 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 21:07:36.115908 10219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:36.112198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60124 (local address 127.11.254.131:40515)
0812 21:07:36.112367 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0812 21:07:36.112371 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 21:07:36.112382 (+ 11us) server_negotiation.cc:407] Connection header received
0812 21:07:36.112464 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:07:36.112466 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:07:36.112479 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:07:36.112554 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:07:36.112726 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:36.113543 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:36.114933 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:36.115535 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:36.115601 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:07:36.115693 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0812 21:07:36.115781 (+ 88us) server_negotiation.cc:299] Negotiation successful
0812 21:07:36.115820 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":83,"threads_started":1}
I20260812 21:07:36.115907 10217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:36.111856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.131:40515 (local address 127.0.0.1:60124)
0812 21:07:36.112329 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0812 21:07:36.112340 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 21:07:36.112397 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:07:36.112562 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:07:36.112563 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:07:36.112565 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:07:36.112644 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:07:36.112647 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:07:36.113614 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:07:36.113615 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:07:36.114842 (+ 1227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:07:36.114845 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:07:36.115607 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:07:36.115608 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:07:36.115684 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:07:36.115698 (+ 14us) client_negotiation.cc:769] Sending connection context
0812 21:07:36.115767 (+ 69us) client_negotiation.cc:240] Negotiation successful
0812 21:07:36.115820 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":451,"thread_start_us":54,"threads_started":1}
I20260812 21:07:36.116438 10213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:36.112033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.130:45765 (local address 127.0.0.1:50916)
0812 21:07:36.112368 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0812 21:07:36.112376 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0812 21:07:36.112414 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:07:36.112897 (+ 483us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:07:36.112898 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:07:36.112901 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:07:36.112936 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:07:36.112938 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:07:36.114159 (+ 1221us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:07:36.114160 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:07:36.115119 (+ 959us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:07:36.115122 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:07:36.116218 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:07:36.116219 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:07:36.116281 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:07:36.116286 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 21:07:36.116335 (+ 49us) client_negotiation.cc:240] Negotiation successful
0812 21:07:36.116374 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":324,"thread_start_us":91,"threads_started":1}
I20260812 21:07:36.116663 10210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:36.111846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.129:44791 (local address 127.0.0.1:52088)
0812 21:07:36.112142 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0812 21:07:36.112183 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 21:07:36.112255 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:07:36.112536 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:07:36.112539 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:07:36.112541 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:07:36.112646 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:07:36.112648 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:07:36.113915 (+ 1267us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:07:36.113916 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:07:36.114972 (+ 1056us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:07:36.114974 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:07:36.116418 (+ 1444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:07:36.116419 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:07:36.116475 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:07:36.116478 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 21:07:36.116521 (+ 43us) client_negotiation.cc:240] Negotiation successful
0812 21:07:36.116604 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":54,"threads_started":1}
I20260812 21:07:36.116670 10218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:36.112153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50916 (local address 127.11.254.130:45765)
0812 21:07:36.112682 (+ 529us) server_negotiation.cc:206] Beginning negotiation
0812 21:07:36.112687 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 21:07:36.112698 (+ 11us) server_negotiation.cc:407] Connection header received
0812 21:07:36.112742 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:07:36.112744 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:07:36.112760 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:07:36.112828 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:07:36.113018 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:36.114091 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:36.115204 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:36.116126 (+ 922us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:36.116188 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:07:36.116273 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0812 21:07:36.116542 (+ 269us) server_negotiation.cc:299] Negotiation successful
0812 21:07:36.116586 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":332,"thread_start_us":86,"threads_started":1}
I20260812 21:07:36.116735 10214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:36.111951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52088 (local address 127.11.254.129:44791)
0812 21:07:36.112150 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0812 21:07:36.112155 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 21:07:36.112250 (+ 95us) server_negotiation.cc:407] Connection header received
0812 21:07:36.112373 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:07:36.112376 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:07:36.112391 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:07:36.112457 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:07:36.112732 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:36.113830 (+ 1098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:36.115198 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:36.115938 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:36.116410 (+ 472us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:07:36.116576 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0812 21:07:36.116602 (+ 26us) server_negotiation.cc:299] Negotiation successful
0812 21:07:36.116653 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":88,"threads_started":1}
I20260812 21:07:36.116806 10094 tablet_service.cc:1467] Processing CreateTablet for tablet 7bbc7c126405482988fc2651d86ea79b (DEFAULT_TABLE table=test-table [id=2f4066681d8044a18ab91ce51be1850f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:07:36.117184 10094 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7bbc7c126405482988fc2651d86ea79b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:07:36.117329 9642 tablet_service.cc:1467] Processing CreateTablet for tablet 7bbc7c126405482988fc2651d86ea79b (DEFAULT_TABLE table=test-table [id=2f4066681d8044a18ab91ce51be1850f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:07:36.117604 9642 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7bbc7c126405482988fc2651d86ea79b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:07:36.119947 10220 tablet_bootstrap.cc:492] T 7bbc7c126405482988fc2651d86ea79b P e598baf6017a4899a58a037fd0afca4d: Bootstrap starting.
I20260812 21:07:36.120471 10221 tablet_bootstrap.cc:492] T 7bbc7c126405482988fc2651d86ea79b P 16b5a2eec7814f65bc01fd3526747fc1: Bootstrap starting.
I20260812 21:07:36.120812 10220 tablet_bootstrap.cc:654] T 7bbc7c126405482988fc2651d86ea79b P e598baf6017a4899a58a037fd0afca4d: Neither blocks nor log segments found. Creating new log.
I20260812 21:07:36.120970 10221 tablet_bootstrap.cc:654] T 7bbc7c126405482988fc2651d86ea79b P 16b5a2eec7814f65bc01fd3526747fc1: Neither blocks nor log segments found. Creating new log.
I20260812 21:07:36.121148 10220 log.cc:824] T 7bbc7c126405482988fc2651d86ea79b P e598baf6017a4899a58a037fd0afca4d: Log is configured to *not* fsync() on all Append() calls
I20260812 21:07:36.121061 9881 tablet_service.cc:1467] Processing CreateTablet for tablet 7bbc7c126405482988fc2651d86ea79b (DEFAULT_TABLE table=test-table [id=2f4066681d8044a18ab91ce51be1850f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:07:36.121188 10221 log.cc:824] T 7bbc7c126405482988fc2651d86ea79b P 16b5a2eec7814f65bc01fd3526747fc1: Log is configured to *not* fsync() on all Append() calls
I20260812 21:07:36.121408 9881 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7bbc7c126405482988fc2651d86ea79b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:07:36.121855 10221 tablet_bootstrap.cc:492] T 7bbc7c126405482988fc2651d86ea79b P 16b5a2eec7814f65bc01fd3526747fc1: No bootstrap required, opened a new log
I20260812 21:07:36.121855 10220 tablet_bootstrap.cc:492] T 7bbc7c126405482988fc2651d86ea79b P e598baf6017a4899a58a037fd0afca4d: No bootstrap required, opened a new log
I20260812 21:07:36.121913 10220 ts_tablet_manager.cc:1397] T 7bbc7c126405482988fc2651d86ea79b P e598baf6017a4899a58a037fd0afca4d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 21:07:36.121913 10221 ts_tablet_manager.cc:1397] T 7bbc7c126405482988fc2651d86ea79b P 16b5a2eec7814f65bc01fd3526747fc1: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 21:07:36.124022 10225 tablet_bootstrap.cc:492] T 7bbc7c126405482988fc2651d86ea79b P 72ce0f1234f14bccb39dd34d942fc51c: Bootstrap starting.
I20260812 21:07:36.124439 10221 raft_consensus.cc:348] T 7bbc7c126405482988fc2651d86ea79b P 16b5a2eec7814f65bc01fd3526747fc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16b5a2eec7814f65bc01fd3526747fc1" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 44791 } } peers { permanent_uuid: "e598baf6017a4899a58a037fd0afca4d" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 40515 } } peers { permanent_uuid: "72ce0f1234f14bccb39dd34d942fc51c" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 45765 } }
I20260812 21:07:36.124432 10220 raft_consensus.cc:348] T 7bbc7c126405482988fc2651d86ea79b P e598baf6017a4899a58a037fd0afca4d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16b5a2eec7814f65bc01fd3526747fc1" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 44791 } } peers { permanent_uuid: "e598baf6017a4899a58a037fd0afca4d" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 40515 } } peers { permanent_uuid: "72ce0f1234f14bccb39dd34d942fc51c" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 45765 } }
I20260812 21:07:36.124615 10221 raft_consensus.cc:374] T 7bbc7c126405482988fc2651d86ea79b P 16b5a2eec7814f65bc01fd3526747fc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:07:36.124615 10220 raft_consensus.cc:374] T 7bbc7c126405482988fc2651d86ea79b P e598baf6017a4899a58a037fd0afca4d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:07:36.124647 10225 tablet_bootstrap.cc:654] T 7bbc7c126405482988fc2651d86ea79b P 72ce0f1234f14bccb39dd34d942fc51c: Neither blocks nor log segments found. Creating new log.
I20260812 21:07:36.124652 10221 raft_consensus.cc:729] T 7bbc7c126405482988fc2651d86ea79b P 16b5a2eec7814f65bc01fd3526747fc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16b5a2eec7814f65bc01fd3526747fc1, State: Initialized, Role: FOLLOWER
I20260812 21:07:36.124652 10220 raft_consensus.cc:729] T 7bbc7c126405482988fc2651d86ea79b P e598baf6017a4899a58a037fd0afca4d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e598baf6017a4899a58a037fd0afca4d, State: Initialized, Role: FOLLOWER
I20260812 21:07:36.124791 10220 consensus_queue.cc:260] T 7bbc7c126405482988fc2651d86ea79b P e598baf6017a4899a58a037fd0afca4d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16b5a2eec7814f65bc01fd3526747fc1" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 44791 } } peers { permanent_uuid: "e598baf6017a4899a58a037fd0afca4d" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 40515 } } peers { permanent_uuid: "72ce0f1234f14bccb39dd34d942fc51c" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 45765 } }
I20260812 21:07:36.124791 10221 consensus_queue.cc:260] T 7bbc7c126405482988fc2651d86ea79b P 16b5a2eec7814f65bc01fd3526747fc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16b5a2eec7814f65bc01fd3526747fc1" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 44791 } } peers { permanent_uuid: "e598baf6017a4899a58a037fd0afca4d" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 40515 } } peers { permanent_uuid: "72ce0f1234f14bccb39dd34d942fc51c" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 45765 } }
I20260812 21:07:36.124951 10225 log.cc:824] T 7bbc7c126405482988fc2651d86ea79b P 72ce0f1234f14bccb39dd34d942fc51c: Log is configured to *not* fsync() on all Append() calls
I20260812 21:07:36.125056 10220 ts_tablet_manager.cc:1428] T 7bbc7c126405482988fc2651d86ea79b P e598baf6017a4899a58a037fd0afca4d: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 21:07:36.125065 10221 ts_tablet_manager.cc:1428] T 7bbc7c126405482988fc2651d86ea79b P 16b5a2eec7814f65bc01fd3526747fc1: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 21:07:36.125182 9766 heartbeater.cc:503] Master 127.11.254.190:32907 was elected leader, sending a full tablet report...
I20260812 21:07:36.125269 10198 heartbeater.cc:503] Master 127.11.254.190:32907 was elected leader, sending a full tablet report...
I20260812 21:07:36.125703 10225 tablet_bootstrap.cc:492] T 7bbc7c126405482988fc2651d86ea79b P 72ce0f1234f14bccb39dd34d942fc51c: No bootstrap required, opened a new log
I20260812 21:07:36.125771 10225 ts_tablet_manager.cc:1397] T 7bbc7c126405482988fc2651d86ea79b P 72ce0f1234f14bccb39dd34d942fc51c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 21:07:36.127910 10225 raft_consensus.cc:348] T 7bbc7c126405482988fc2651d86ea79b P 72ce0f1234f14bccb39dd34d942fc51c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16b5a2eec7814f65bc01fd3526747fc1" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 44791 } } peers { permanent_uuid: "e598baf6017a4899a58a037fd0afca4d" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 40515 } } peers { permanent_uuid: "72ce0f1234f14bccb39dd34d942fc51c" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 45765 } }
I20260812 21:07:36.128062 10225 raft_consensus.cc:374] T 7bbc7c126405482988fc2651d86ea79b P 72ce0f1234f14bccb39dd34d942fc51c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:07:36.128093 10225 raft_consensus.cc:729] T 7bbc7c126405482988fc2651d86ea79b P 72ce0f1234f14bccb39dd34d942fc51c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72ce0f1234f14bccb39dd34d942fc51c, State: Initialized, Role: FOLLOWER
I20260812 21:07:36.128207 10225 consensus_queue.cc:260] T 7bbc7c126405482988fc2651d86ea79b P 72ce0f1234f14bccb39dd34d942fc51c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16b5a2eec7814f65bc01fd3526747fc1" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 44791 } } peers { permanent_uuid: "e598baf6017a4899a58a037fd0afca4d" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 40515 } } peers { permanent_uuid: "72ce0f1234f14bccb39dd34d942fc51c" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 45765 } }
I20260812 21:07:36.128443 10225 ts_tablet_manager.cc:1428] T 7bbc7c126405482988fc2651d86ea79b P 72ce0f1234f14bccb39dd34d942fc51c: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 21:07:36.128504 9986 heartbeater.cc:503] Master 127.11.254.190:32907 was elected leader, sending a full tablet report...
I20260812 21:07:36.134944 10226 raft_consensus.cc:482] T 7bbc7c126405482988fc2651d86ea79b P e598baf6017a4899a58a037fd0afca4d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 21:07:36.135073 10226 raft_consensus.cc:504] T 7bbc7c126405482988fc2651d86ea79b P e598baf6017a4899a58a037fd0afca4d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16b5a2eec7814f65bc01fd3526747fc1" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 44791 } } peers { permanent_uuid: "e598baf6017a4899a58a037fd0afca4d" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 40515 } } peers { permanent_uuid: "72ce0f1234f14bccb39dd34d942fc51c" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 45765 } }
I20260812 21:07:36.135447 10226 leader_election.cc:288] T 7bbc7c126405482988fc2651d86ea79b P e598baf6017a4899a58a037fd0afca4d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 16b5a2eec7814f65bc01fd3526747fc1 (127.11.254.129:44791), 72ce0f1234f14bccb39dd34d942fc51c (127.11.254.130:45765)
I20260812 21:07:36.138388 10214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:36.135694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.131:34441 (local address 127.11.254.129:44791)
0812 21:07:36.135755 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0812 21:07:36.135757 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:07:36.135899 (+ 142us) server_negotiation.cc:407] Connection header received
0812 21:07:36.135978 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:07:36.135979 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:07:36.135991 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:07:36.136015 (+ 24us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:07:36.136205 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:36.136727 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:36.137669 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:36.138083 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:36.138151 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:07:36.138159 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 21:07:36.138286 (+ 127us) server_negotiation.cc:299] Negotiation successful
0812 21:07:36.138321 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260812 21:07:36.138392 10201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:36.135614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.130:45765 (local address 127.11.254.131:50897)
0812 21:07:36.135654 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0812 21:07:36.135668 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 21:07:36.135712 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:07:36.135955 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:07:36.135956 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:07:36.135958 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:07:36.136057 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:07:36.136060 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:07:36.136744 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:07:36.136745 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:07:36.137575 (+ 830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:07:36.137577 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:07:36.138142 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:07:36.138143 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:07:36.138215 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:07:36.138219 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 21:07:36.138261 (+ 42us) client_negotiation.cc:240] Negotiation successful
0812 21:07:36.138306 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19,"mutex_wait_us":42}
I20260812 21:07:36.138384 10218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:36.135776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.131:50897 (local address 127.11.254.130:45765)
0812 21:07:36.135835 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0812 21:07:36.135837 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:07:36.135845 (+ 8us) server_negotiation.cc:407] Connection header received
0812 21:07:36.135872 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:07:36.135873 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:07:36.135884 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:07:36.135935 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:07:36.136140 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:36.136666 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:36.137658 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:36.138070 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:36.138127 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:07:36.138136 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 21:07:36.138274 (+ 138us) server_negotiation.cc:299] Negotiation successful
0812 21:07:36.138310 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260812 21:07:36.138409 10230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:36.135619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.129:44791 (local address 127.11.254.131:34441)
0812 21:07:36.135827 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0812 21:07:36.135837 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 21:07:36.135892 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:07:36.136069 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:07:36.136070 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:07:36.136071 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:07:36.136137 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:07:36.136139 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:07:36.136805 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:07:36.136806 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:07:36.137587 (+ 781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:07:36.137589 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:07:36.138163 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:07:36.138164 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:07:36.138222 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:07:36.138226 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 21:07:36.138274 (+ 48us) client_negotiation.cc:240] Negotiation successful
0812 21:07:36.138313 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":191,"mutex_wait_us":18,"thread_start_us":115,"threads_started":1}
I20260812 21:07:36.138764 9913 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7bbc7c126405482988fc2651d86ea79b" candidate_uuid: "e598baf6017a4899a58a037fd0afca4d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72ce0f1234f14bccb39dd34d942fc51c" is_pre_election: true
I20260812 21:07:36.138803 9683 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7bbc7c126405482988fc2651d86ea79b" candidate_uuid: "e598baf6017a4899a58a037fd0afca4d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16b5a2eec7814f65bc01fd3526747fc1" is_pre_election: true
I20260812 21:07:36.138933 9683 raft_consensus.cc:2454] T 7bbc7c126405482988fc2651d86ea79b P 16b5a2eec7814f65bc01fd3526747fc1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e598baf6017a4899a58a037fd0afca4d in term 0.
I20260812 21:07:36.138933 9913 raft_consensus.cc:2454] T 7bbc7c126405482988fc2651d86ea79b P 72ce0f1234f14bccb39dd34d942fc51c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e598baf6017a4899a58a037fd0afca4d in term 0.
I20260812 21:07:36.139166 10013 leader_election.cc:302] T 7bbc7c126405482988fc2651d86ea79b P e598baf6017a4899a58a037fd0afca4d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16b5a2eec7814f65bc01fd3526747fc1, e598baf6017a4899a58a037fd0afca4d; no voters:
I20260812 21:07:36.139304 10226 raft_consensus.cc:2781] T 7bbc7c126405482988fc2651d86ea79b P e598baf6017a4899a58a037fd0afca4d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 21:07:36.139374 10226 raft_consensus.cc:482] T 7bbc7c126405482988fc2651d86ea79b P e598baf6017a4899a58a037fd0afca4d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 21:07:36.139400 10226 raft_consensus.cc:3037] T 7bbc7c126405482988fc2651d86ea79b P e598baf6017a4899a58a037fd0afca4d [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:07:36.140126 10226 raft_consensus.cc:504] T 7bbc7c126405482988fc2651d86ea79b P e598baf6017a4899a58a037fd0afca4d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16b5a2eec7814f65bc01fd3526747fc1" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 44791 } } peers { permanent_uuid: "e598baf6017a4899a58a037fd0afca4d" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 40515 } } peers { permanent_uuid: "72ce0f1234f14bccb39dd34d942fc51c" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 45765 } }
I20260812 21:07:36.140251 10226 leader_election.cc:288] T 7bbc7c126405482988fc2651d86ea79b P e598baf6017a4899a58a037fd0afca4d [CANDIDATE]: Term 1 election: Requested vote from peers 16b5a2eec7814f65bc01fd3526747fc1 (127.11.254.129:44791), 72ce0f1234f14bccb39dd34d942fc51c (127.11.254.130:45765)
I20260812 21:07:36.140399 9913 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7bbc7c126405482988fc2651d86ea79b" candidate_uuid: "e598baf6017a4899a58a037fd0afca4d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72ce0f1234f14bccb39dd34d942fc51c"
I20260812 21:07:36.140408 9683 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7bbc7c126405482988fc2651d86ea79b" candidate_uuid: "e598baf6017a4899a58a037fd0afca4d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16b5a2eec7814f65bc01fd3526747fc1"
I20260812 21:07:36.140477 9683 raft_consensus.cc:3037] T 7bbc7c126405482988fc2651d86ea79b P 16b5a2eec7814f65bc01fd3526747fc1 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:07:36.140486 9913 raft_consensus.cc:3037] T 7bbc7c126405482988fc2651d86ea79b P 72ce0f1234f14bccb39dd34d942fc51c [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:07:36.141352 9913 raft_consensus.cc:2454] T 7bbc7c126405482988fc2651d86ea79b P 72ce0f1234f14bccb39dd34d942fc51c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e598baf6017a4899a58a037fd0afca4d in term 1.
I20260812 21:07:36.141352 9683 raft_consensus.cc:2454] T 7bbc7c126405482988fc2651d86ea79b P 16b5a2eec7814f65bc01fd3526747fc1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e598baf6017a4899a58a037fd0afca4d in term 1.
I20260812 21:07:36.141566 10017 leader_election.cc:302] T 7bbc7c126405482988fc2651d86ea79b P e598baf6017a4899a58a037fd0afca4d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 72ce0f1234f14bccb39dd34d942fc51c, e598baf6017a4899a58a037fd0afca4d; no voters:
I20260812 21:07:36.141680 10226 raft_consensus.cc:2781] T 7bbc7c126405482988fc2651d86ea79b P e598baf6017a4899a58a037fd0afca4d [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:07:36.141824 10226 raft_consensus.cc:686] T 7bbc7c126405482988fc2651d86ea79b P e598baf6017a4899a58a037fd0afca4d [term 1 LEADER]: Becoming Leader. State: Replica: e598baf6017a4899a58a037fd0afca4d, State: Running, Role: LEADER
I20260812 21:07:36.141927 10226 consensus_queue.cc:237] T 7bbc7c126405482988fc2651d86ea79b P e598baf6017a4899a58a037fd0afca4d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16b5a2eec7814f65bc01fd3526747fc1" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 44791 } } peers { permanent_uuid: "e598baf6017a4899a58a037fd0afca4d" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 40515 } } peers { permanent_uuid: "72ce0f1234f14bccb39dd34d942fc51c" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 45765 } }
I20260812 21:07:36.142819 9432 catalog_manager.cc:5495] T 7bbc7c126405482988fc2651d86ea79b P e598baf6017a4899a58a037fd0afca4d reported cstate change: term changed from 0 to 1, leader changed from <none> to e598baf6017a4899a58a037fd0afca4d (127.11.254.131). New cstate: current_term: 1 leader_uuid: "e598baf6017a4899a58a037fd0afca4d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16b5a2eec7814f65bc01fd3526747fc1" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 44791 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e598baf6017a4899a58a037fd0afca4d" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 40515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "72ce0f1234f14bccb39dd34d942fc51c" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 45765 } health_report { overall_health: UNKNOWN } } }
I20260812 21:07:36.151168 10219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:36.149011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60126 (local address 127.11.254.131:40515)
0812 21:07:36.149051 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0812 21:07:36.149052 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 21:07:36.149061 (+ 9us) server_negotiation.cc:407] Connection header received
0812 21:07:36.149121 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:07:36.149122 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:07:36.149136 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:07:36.149192 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 21:07:36.149288 (+ 96us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:36.149733 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:36.150494 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:36.150691 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:36.150763 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:07:36.150906 (+ 143us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 21:07:36.151000 (+ 94us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 21:07:36.151062 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0812 21:07:36.151076 (+ 14us) server_negotiation.cc:299] Negotiation successful
0812 21:07:36.151093 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260812 21:07:36.153034 9913 raft_consensus.cc:1261] T 7bbc7c126405482988fc2651d86ea79b P 72ce0f1234f14bccb39dd34d942fc51c [term 1 FOLLOWER]: Refusing update from remote peer e598baf6017a4899a58a037fd0afca4d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 21:07:36.153043 9683 raft_consensus.cc:1261] T 7bbc7c126405482988fc2651d86ea79b P 16b5a2eec7814f65bc01fd3526747fc1 [term 1 FOLLOWER]: Refusing update from remote peer e598baf6017a4899a58a037fd0afca4d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 21:07:36.153369 10231 consensus_queue.cc:1035] T 7bbc7c126405482988fc2651d86ea79b P e598baf6017a4899a58a037fd0afca4d [LEADER]: Connected to new peer: Peer: permanent_uuid: "72ce0f1234f14bccb39dd34d942fc51c" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 45765 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 21:07:36.153499 10226 consensus_queue.cc:1035] T 7bbc7c126405482988fc2651d86ea79b P e598baf6017a4899a58a037fd0afca4d [LEADER]: Connected to new peer: Peer: permanent_uuid: "16b5a2eec7814f65bc01fd3526747fc1" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 44791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 21:07:36.155766 10236 mvcc.cc:205] Tried to move back new op lower bound from 7317786034800058368 to 7317786034758189056. Current Snapshot: MvccSnapshot[applied={T|T < 7317786034800058368}]
I20260812 21:07:36.156132 10238 mvcc.cc:205] Tried to move back new op lower bound from 7317786034800058368 to 7317786034758189056. Current Snapshot: MvccSnapshot[applied={T|T < 7317786034800058368}]
I20260812 21:07:36.158285 9432 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:55512:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 21:07:36.158370 9432 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:55512:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 21:07:36.159082 9432 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d3f0ec64844d40d98e0bd0ba697e774d: Sending DeleteTablet for 3 replicas of tablet 7bbc7c126405482988fc2651d86ea79b
I20260812 21:07:36.159454 9642 tablet_service.cc:1514] Processing DeleteTablet for tablet 7bbc7c126405482988fc2651d86ea79b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:07:36 UTC) from {username='slave'} at 127.0.0.1:52088
I20260812 21:07:36.159469 10094 tablet_service.cc:1514] Processing DeleteTablet for tablet 7bbc7c126405482988fc2651d86ea79b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:07:36 UTC) from {username='slave'} at 127.0.0.1:60124
I20260812 21:07:36.159703 10249 tablet_replica.cc:330] T 7bbc7c126405482988fc2651d86ea79b P 16b5a2eec7814f65bc01fd3526747fc1: stopping tablet replica
I20260812 21:07:36.159705 9881 tablet_service.cc:1514] Processing DeleteTablet for tablet 7bbc7c126405482988fc2651d86ea79b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:07:36 UTC) from {username='slave'} at 127.0.0.1:50916
I20260812 21:07:36.159811 10249 raft_consensus.cc:2229] T 7bbc7c126405482988fc2651d86ea79b P 16b5a2eec7814f65bc01fd3526747fc1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 21:07:36.159919 10249 raft_consensus.cc:2258] T 7bbc7c126405482988fc2651d86ea79b P 16b5a2eec7814f65bc01fd3526747fc1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 21:07:36.159998 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 9523
I20260812 21:07:36.160008 10251 tablet_replica.cc:330] T 7bbc7c126405482988fc2651d86ea79b P 72ce0f1234f14bccb39dd34d942fc51c: stopping tablet replica
I20260812 21:07:36.173758 10251 raft_consensus.cc:2229] T 7bbc7c126405482988fc2651d86ea79b P 72ce0f1234f14bccb39dd34d942fc51c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 21:07:36.174155 10251 raft_consensus.cc:2258] T 7bbc7c126405482988fc2651d86ea79b P 72ce0f1234f14bccb39dd34d942fc51c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 21:07:36.175000 10251 ts_tablet_manager.cc:1905] T 7bbc7c126405482988fc2651d86ea79b P 72ce0f1234f14bccb39dd34d942fc51c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 21:07:36.176755 10251 ts_tablet_manager.cc:1918] T 7bbc7c126405482988fc2651d86ea79b P 72ce0f1234f14bccb39dd34d942fc51c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 21:07:36.176934 10251 log.cc:1196] T 7bbc7c126405482988fc2651d86ea79b P 72ce0f1234f14bccb39dd34d942fc51c: Deleting WAL directory at /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786568607354921-12282-0/minicluster-data/ts-1/wal/wals/7bbc7c126405482988fc2651d86ea79b
I20260812 21:07:36.177340 10251 ts_tablet_manager.cc:1939] T 7bbc7c126405482988fc2651d86ea79b P 72ce0f1234f14bccb39dd34d942fc51c: Deleting consensus metadata
I20260812 21:07:36.178009 9408 catalog_manager.cc:4841] TS 72ce0f1234f14bccb39dd34d942fc51c (127.11.254.130:45765): tablet 7bbc7c126405482988fc2651d86ea79b (table test-table [id=2f4066681d8044a18ab91ce51be1850f]) successfully deleted
I20260812 21:07:36.186985 10250 tablet_replica.cc:330] T 7bbc7c126405482988fc2651d86ea79b P e598baf6017a4899a58a037fd0afca4d: stopping tablet replica
I20260812 21:07:36.187170 10250 raft_consensus.cc:2229] T 7bbc7c126405482988fc2651d86ea79b P e598baf6017a4899a58a037fd0afca4d [term 1 LEADER]: Raft consensus shutting down.
I20260812 21:07:36.187345 10250 raft_consensus.cc:2258] T 7bbc7c126405482988fc2651d86ea79b P e598baf6017a4899a58a037fd0afca4d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 21:07:36.187920 10250 ts_tablet_manager.cc:1905] T 7bbc7c126405482988fc2651d86ea79b P e598baf6017a4899a58a037fd0afca4d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 21:07:36.189469 10250 ts_tablet_manager.cc:1918] T 7bbc7c126405482988fc2651d86ea79b P e598baf6017a4899a58a037fd0afca4d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 21:07:36.189556 10250 log.cc:1196] T 7bbc7c126405482988fc2651d86ea79b P e598baf6017a4899a58a037fd0afca4d: Deleting WAL directory at /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786568607354921-12282-0/minicluster-data/ts-2/wal/wals/7bbc7c126405482988fc2651d86ea79b
I20260812 21:07:36.189786 10250 ts_tablet_manager.cc:1939] T 7bbc7c126405482988fc2651d86ea79b P e598baf6017a4899a58a037fd0afca4d: Deleting consensus metadata
I20260812 21:07:36.190256 9408 catalog_manager.cc:4841] TS e598baf6017a4899a58a037fd0afca4d (127.11.254.131:40515): tablet 7bbc7c126405482988fc2651d86ea79b (table test-table [id=2f4066681d8044a18ab91ce51be1850f]) successfully deleted
W20260812 21:07:36.193619 9407 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.254.129:44791 (error 108)
I20260812 21:07:36.193873 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 9772
I20260812 21:07:36.196671 10210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:36.194023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.129:44791 (local address 127.0.0.1:52094)
0812 21:07:36.196573 (+ 2550us) negotiation.cc:107] Waiting for socket to connect
0812 21:07:36.196636 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.129:44791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2522}
W20260812 21:07:36.201357 9407 catalog_manager.cc:4584] TS 16b5a2eec7814f65bc01fd3526747fc1 (127.11.254.129:44791): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7bbc7c126405482988fc2651d86ea79b: Network error: Client connection negotiation failed: client connection to 127.11.254.129:44791: connect: Connection refused (error 111)
I20260812 21:07:36.206612 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 9990
I20260812 21:07:36.223680 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 9374
2026-08-12T21:07:36Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (772 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568607354921-12282-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 21:07:36 dist-test-slave-vpvm krb5kdc[10264](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 21:07:36 dist-test-slave-vpvm krb5kdc[10264](info): set up 2 sockets
Aug 12 21:07:36 dist-test-slave-vpvm krb5kdc[10264](info): commencing operation
krb5kdc: starting...
W20260812 21:07:37.404265 12282 mini_kdc.cc:121] Time spent starting KDC: real 1.153s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568607354921-12282-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568607354921-12282-0/krb5kdc/test-user.keytab.
Aug 12 21:07:37 dist-test-slave-vpvm krb5kdc[10264](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568857, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T21:07:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T21:07:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.254.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.254.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.254.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.190.keytab.
Entry for principal kudu/127.11.254.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.190.keytab.
I20260812 21:07:37.536626 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568607354921-12282-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568607354921-12282-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568607354921-12282-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.254.190:44439
--webserver_interface=127.11.254.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.190.keytab
--principal=kudu/127.11.254.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.254.148:34715
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.254.190:44439
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568607354921-12282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568607354921-12282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568607354921-12282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:07:37.620056 10285 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:07:37.620280 10285 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:07:37.620311 10285 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:07:37.621477 10285 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 21:07:37.621524 10285 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:07:37.621541 10285 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:07:37.621558 10285 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 21:07:37.621573 10285 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 21:07:37.623107 10285 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.254.148:34715
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568607354921-12282-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568607354921-12282-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.254.190:44439
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.190.keytab
--principal=kudu/127.11.254.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.254.190:44439
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.254.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10285
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568607354921-12282-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:07:37.623374 10285 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:07:37.623588 10285 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:07:37.628433 10307 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:07:37.628425 10299 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:07:37.628458 10285 server_base.cc:1034] running on GCE node
W20260812 21:07:37.628433 10298 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:07:37.628924 10285 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:07:37.629213 10285 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:07:37.630380 10285 hybrid_clock.cc:648] HybridClock initialized: now 1786568857630365 us; error 29 us; skew 500 ppm
Aug 12 21:07:37 dist-test-slave-vpvm krb5kdc[10264](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568857, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:07:37.641438 10285 init.cc:377] Logged in from keytab as kudu/127.11.254.190@KRBTEST.COM (short username kudu)
I20260812 21:07:37.642027 10285 webserver.cc:460] Webserver started at http://127.11.254.190:43315/ using document root <none> and password file <none>
I20260812 21:07:37.642257 10285 fs_manager.cc:359] Metadata directory not provided
I20260812 21:07:37.642304 10285 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:07:37.642431 10285 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:07:37.643548 10285 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568607354921-12282-0/minicluster-data/master-0/data/instance:
uuid: "ee3e88fa99e040f587bf2da48520d7c1"
format_stamp: "Formatted at 2026-08-12 21:07:37 on dist-test-slave-vpvm"
server_key: "b51bbf9746ab09fc53af3335adfc0b80"
server_key_iv: "7334a28132f30885096723b110ef25b9"
server_key_version: "encryptionkey@0"
I20260812 21:07:37.643898 10285 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568607354921-12282-0/minicluster-data/master-0/wal/instance:
uuid: "ee3e88fa99e040f587bf2da48520d7c1"
format_stamp: "Formatted at 2026-08-12 21:07:37 on dist-test-slave-vpvm"
server_key: "b51bbf9746ab09fc53af3335adfc0b80"
server_key_iv: "7334a28132f30885096723b110ef25b9"
server_key_version: "encryptionkey@0"
I20260812 21:07:37.645265 10285 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 21:07:37.646142 10320 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:07:37.646327 10285 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 21:07:37.646425 10285 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568607354921-12282-0/minicluster-data/master-0/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568607354921-12282-0/minicluster-data/master-0/wal
uuid: "ee3e88fa99e040f587bf2da48520d7c1"
format_stamp: "Formatted at 2026-08-12 21:07:37 on dist-test-slave-vpvm"
server_key: "b51bbf9746ab09fc53af3335adfc0b80"
server_key_iv: "7334a28132f30885096723b110ef25b9"
server_key_version: "encryptionkey@0"
I20260812 21:07:37.646553 10285 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568607354921-12282-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568607354921-12282-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568607354921-12282-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:07:37.669772 10285 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:07:37.672127 10285 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:07:37.672279 10285 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:07:37.676398 10285 rpc_server.cc:307] RPC server started. Bound to: 127.11.254.190:44439
I20260812 21:07:37.676436 10405 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.254.190:44439 every 8 connection(s)
I20260812 21:07:37.676791 10285 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
I20260812 21:07:37.677660 10406 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:07:37.679996 10406 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ee3e88fa99e040f587bf2da48520d7c1: Bootstrap starting.
I20260812 21:07:37.680563 10406 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ee3e88fa99e040f587bf2da48520d7c1: Neither blocks nor log segments found. Creating new log.
I20260812 21:07:37.680791 10406 log.cc:824] T 00000000000000000000000000000000 P ee3e88fa99e040f587bf2da48520d7c1: Log is configured to *not* fsync() on all Append() calls
I20260812 21:07:37.681458 10406 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ee3e88fa99e040f587bf2da48520d7c1: No bootstrap required, opened a new log
I20260812 21:07:37.682956 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 10285
I20260812 21:07:37.683063 12282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568607354921-12282-0/minicluster-data/master-0/wal/instance
I20260812 21:07:37.683259 12282 external_mini_cluster.cc:1428] Setting key 9f3195bd6c8123d67985191f87d621aa
I20260812 21:07:37.683244 10406 raft_consensus.cc:348] T 00000000000000000000000000000000 P ee3e88fa99e040f587bf2da48520d7c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee3e88fa99e040f587bf2da48520d7c1" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 44439 } }
I20260812 21:07:37.683399 10406 raft_consensus.cc:374] T 00000000000000000000000000000000 P ee3e88fa99e040f587bf2da48520d7c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:07:37.683445 10406 raft_consensus.cc:729] T 00000000000000000000000000000000 P ee3e88fa99e040f587bf2da48520d7c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee3e88fa99e040f587bf2da48520d7c1, State: Initialized, Role: FOLLOWER
I20260812 21:07:37.683589 10406 consensus_queue.cc:260] T 00000000000000000000000000000000 P ee3e88fa99e040f587bf2da48520d7c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee3e88fa99e040f587bf2da48520d7c1" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 44439 } }
I20260812 21:07:37.683677 10406 raft_consensus.cc:388] T 00000000000000000000000000000000 P ee3e88fa99e040f587bf2da48520d7c1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 21:07:37.683707 10406 raft_consensus.cc:482] T 00000000000000000000000000000000 P ee3e88fa99e040f587bf2da48520d7c1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 21:07:37.683748 10406 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ee3e88fa99e040f587bf2da48520d7c1 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:07:37.684427 10406 raft_consensus.cc:504] T 00000000000000000000000000000000 P ee3e88fa99e040f587bf2da48520d7c1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee3e88fa99e040f587bf2da48520d7c1" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 44439 } }
I20260812 21:07:37.684536 10406 leader_election.cc:302] T 00000000000000000000000000000000 P ee3e88fa99e040f587bf2da48520d7c1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ee3e88fa99e040f587bf2da48520d7c1; no voters:
I20260812 21:07:37.684707 10406 leader_election.cc:288] T 00000000000000000000000000000000 P ee3e88fa99e040f587bf2da48520d7c1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 21:07:37.684760 10414 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ee3e88fa99e040f587bf2da48520d7c1 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:07:37.684878 10414 raft_consensus.cc:686] T 00000000000000000000000000000000 P ee3e88fa99e040f587bf2da48520d7c1 [term 1 LEADER]: Becoming Leader. State: Replica: ee3e88fa99e040f587bf2da48520d7c1, State: Running, Role: LEADER
I20260812 21:07:37.684962 10406 sys_catalog.cc:564] T 00000000000000000000000000000000 P ee3e88fa99e040f587bf2da48520d7c1 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 21:07:37.684957 10414 consensus_queue.cc:237] T 00000000000000000000000000000000 P ee3e88fa99e040f587bf2da48520d7c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee3e88fa99e040f587bf2da48520d7c1" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 44439 } }
I20260812 21:07:37.685979 10415 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee3e88fa99e040f587bf2da48520d7c1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ee3e88fa99e040f587bf2da48520d7c1. Latest consensus state: current_term: 1 leader_uuid: "ee3e88fa99e040f587bf2da48520d7c1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee3e88fa99e040f587bf2da48520d7c1" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 44439 } } }
I20260812 21:07:37.686096 10415 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee3e88fa99e040f587bf2da48520d7c1 [sys.catalog]: This master's current role is: LEADER
I20260812 21:07:37.686641 10416 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee3e88fa99e040f587bf2da48520d7c1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ee3e88fa99e040f587bf2da48520d7c1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee3e88fa99e040f587bf2da48520d7c1" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 44439 } } }
I20260812 21:07:37.686735 10416 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee3e88fa99e040f587bf2da48520d7c1 [sys.catalog]: This master's current role is: LEADER
W20260812 21:07:37.687085 10438 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ee3e88fa99e040f587bf2da48520d7c1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 21:07:37.687186 10438 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 21:07:37.687259 10431 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 21:07:37.687433 10431 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 21:07:37.688978 10431 catalog_manager.cc:1347] Generated new cluster ID: 216572c5a6674737828b2e6ba47f7871
I20260812 21:07:37.689035 10431 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 21:07:37 dist-test-slave-vpvm krb5kdc[10264](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568857, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.254.190@KRBTEST.COM
I20260812 21:07:37.694051 10412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:37.683742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54076 (local address 127.11.254.190:44439)
0812 21:07:37.683943 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0812 21:07:37.683953 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 21:07:37.683988 (+ 35us) server_negotiation.cc:407] Connection header received
0812 21:07:37.684386 (+ 398us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:07:37.684396 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:07:37.684461 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:07:37.684626 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:07:37.685478 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:37.686973 (+ 1495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:37.688030 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:37.688637 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:37.688742 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:07:37.691028 (+ 2286us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:07:37.691032 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:07:37.691033 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:07:37.691045 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:07:37.692600 (+ 1555us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:07:37.692934 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:07:37.692937 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:07:37.692937 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:07:37.693038 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:07:37.693226 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:07:37.693227 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:07:37.693228 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:07:37.693471 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:07:37.693635 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0812 21:07:37.693707 (+ 72us) server_negotiation.cc:299] Negotiation successful
0812 21:07:37.693846 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":105,"threads_started":1}
I20260812 21:07:37.699404 10431 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 21:07:37.700039 10431 catalog_manager.cc:1504] Loading token signing keys...
I20260812 21:07:37.704742 10431 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ee3e88fa99e040f587bf2da48520d7c1: Generated new TSK 0
I20260812 21:07:37.704928 10431 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.254.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.254.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.254.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.129.keytab.
Entry for principal kudu/127.11.254.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.129.keytab.
I20260812 21:07:37.738121 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568607354921-12282-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568607354921-12282-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568607354921-12282-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568607354921-12282-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.254.129:0
--local_ip_for_outbound_sockets=127.11.254.129
--webserver_interface=127.11.254.129
--webserver_port=0
--tserver_master_addrs=127.11.254.190:44439
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.129.keytab
--principal=kudu/127.11.254.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.254.148:34715
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568607354921-12282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568607354921-12282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568607354921-12282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:07:37.826546 10448 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:07:37.826766 10448 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:07:37.826800 10448 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:07:37.827973 10448 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:07:37.828014 10448 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:07:37.828051 10448 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.254.129
I20260812 21:07:37.829589 10448 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.254.148:34715
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568607354921-12282-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568607354921-12282-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.129.keytab
--principal=kudu/127.11.254.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.254.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568607354921-12282-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.254.129
--webserver_port=0
--tserver_master_addrs=127.11.254.190:44439
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10448
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.254.129
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568607354921-12282-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:07:37.829839 10448 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:07:37.830094 10448 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:07:37.835115 10459 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:07:37.835175 10456 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:07:37.835110 10455 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:07:37.835578 10448 server_base.cc:1034] running on GCE node
I20260812 21:07:37.835793 10448 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:07:37.836076 10448 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:07:37.837246 10448 hybrid_clock.cc:648] HybridClock initialized: now 1786568857837226 us; error 29 us; skew 500 ppm
Aug 12 21:07:37 dist-test-slave-vpvm krb5kdc[10264](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568857, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:07:37.840035 10448 init.cc:377] Logged in from keytab as kudu/127.11.254.129@KRBTEST.COM (short username kudu)
I20260812 21:07:37.840528 10448 webserver.cc:460] Webserver started at http://127.11.254.129:42461/ using document root <none> and password file <none>
I20260812 21:07:37.840770 10448 fs_manager.cc:359] Metadata directory not provided
I20260812 21:07:37.840821 10448 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:07:37.840966 10448 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:07:37.842135 10448 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568607354921-12282-0/minicluster-data/ts-0/data/instance:
uuid: "790b4220d8394f438938b2236dcafb61"
format_stamp: "Formatted at 2026-08-12 21:07:37 on dist-test-slave-vpvm"
server_key: "ffe87fb412fa5f6e4797026e1835002c"
server_key_iv: "433dd3096f4b5e8a253eacfd17a3e008"
server_key_version: "encryptionkey@0"
I20260812 21:07:37.842499 10448 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568607354921-12282-0/minicluster-data/ts-0/wal/instance:
uuid: "790b4220d8394f438938b2236dcafb61"
format_stamp: "Formatted at 2026-08-12 21:07:37 on dist-test-slave-vpvm"
server_key: "ffe87fb412fa5f6e4797026e1835002c"
server_key_iv: "433dd3096f4b5e8a253eacfd17a3e008"
server_key_version: "encryptionkey@0"
I20260812 21:07:37.843943 10448 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260812 21:07:37.844731 10472 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:07:37.844892 10448 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 21:07:37.844991 10448 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568607354921-12282-0/minicluster-data/ts-0/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568607354921-12282-0/minicluster-data/ts-0/wal
uuid: "790b4220d8394f438938b2236dcafb61"
format_stamp: "Formatted at 2026-08-12 21:07:37 on dist-test-slave-vpvm"
server_key: "ffe87fb412fa5f6e4797026e1835002c"
server_key_iv: "433dd3096f4b5e8a253eacfd17a3e008"
server_key_version: "encryptionkey@0"
I20260812 21:07:37.845057 10448 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568607354921-12282-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568607354921-12282-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568607354921-12282-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:07:37.854741 10448 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:07:37.856870 10448 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:07:37.857010 10448 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:07:37.857231 10448 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:07:37.857538 10448 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:07:37.857573 10448 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:07:37.857602 10448 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:07:37.857626 10448 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:07:37.866025 10448 rpc_server.cc:307] RPC server started. Bound to: 127.11.254.129:34871
I20260812 21:07:37.866086 10662 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.254.129:34871 every 8 connection(s)
I20260812 21:07:37.866447 10448 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568607354921-12282-0/minicluster-data/ts-0/data/info.pb
Aug 12 21:07:37 dist-test-slave-vpvm krb5kdc[10264](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568857, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.129@KRBTEST.COM for kudu/127.11.254.190@KRBTEST.COM
I20260812 21:07:37.874948 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 10448
I20260812 21:07:37.875083 12282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568607354921-12282-0/minicluster-data/ts-0/wal/instance
I20260812 21:07:37.875319 12282 external_mini_cluster.cc:1428] Setting key d5c2559e38d075446dbd2844321f2a06
I20260812 21:07:37.876152 10412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:37.867168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.129:56385 (local address 127.11.254.190:44439)
0812 21:07:37.867250 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0812 21:07:37.867252 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:07:37.867386 (+ 134us) server_negotiation.cc:407] Connection header received
0812 21:07:37.867972 (+ 586us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:07:37.867973 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:07:37.867989 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:07:37.868051 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:07:37.869061 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:37.869574 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:37.870608 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:37.870839 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:37.870900 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:07:37.873038 (+ 2138us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:07:37.873040 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:07:37.873040 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:07:37.873048 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:07:37.874434 (+ 1386us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:07:37.874914 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:07:37.874916 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:07:37.874917 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:07:37.875016 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:07:37.875319 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:07:37.875320 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:07:37.875325 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:07:37.875488 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:07:37.875571 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0812 21:07:37.875974 (+ 403us) server_negotiation.cc:299] Negotiation successful
0812 21:07:37.876085 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 21:07:37.876327 10665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:37.867127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:44439 (local address 127.11.254.129:56385)
0812 21:07:37.867270 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0812 21:07:37.867308 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 21:07:37.867842 (+ 534us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:07:37.868121 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:07:37.868125 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:07:37.868744 (+ 619us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:07:37.868953 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:07:37.868959 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:07:37.869652 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:07:37.869652 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:07:37.870521 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:07:37.870525 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:07:37.870928 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:07:37.870929 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:07:37.871054 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:07:37.871516 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:07:37.871524 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:07:37.872953 (+ 1429us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:07:37.874549 (+ 1596us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:07:37.874557 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:07:37.874559 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:07:37.874820 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:07:37.875126 (+ 306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:07:37.875127 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:07:37.875127 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:07:37.875250 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:07:37.875586 (+ 336us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:07:37.875595 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:07:37.875769 (+ 174us) client_negotiation.cc:769] Sending connection context
0812 21:07:37.875888 (+ 119us) client_negotiation.cc:240] Negotiation successful
0812 21:07:37.876074 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":72,"threads_started":1}
I20260812 21:07:37.877121 10663 heartbeater.cc:347] Connected to a master server at 127.11.254.190:44439
I20260812 21:07:37.877246 10663 heartbeater.cc:464] Registering TS with master...
I20260812 21:07:37.877447 10663 heartbeater.cc:511] Master 127.11.254.190:44439 requested a full tablet report, sending...
I20260812 21:07:37.878068 10350 ts_manager.cc:194] Registered new tserver with Master: 790b4220d8394f438938b2236dcafb61 (127.11.254.129:34871)
I20260812 21:07:37.878713 10350 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.254.129@KRBTEST.COM'} at 127.11.254.129:56385
WARNING: no policy specified for kudu/127.11.254.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.254.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.254.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.130.keytab.
Entry for principal kudu/127.11.254.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.130.keytab.
I20260812 21:07:37.906105 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568607354921-12282-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568607354921-12282-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568607354921-12282-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568607354921-12282-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.254.130:0
--local_ip_for_outbound_sockets=127.11.254.130
--webserver_interface=127.11.254.130
--webserver_port=0
--tserver_master_addrs=127.11.254.190:44439
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.130.keytab
--principal=kudu/127.11.254.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.254.148:34715
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568607354921-12282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568607354921-12282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568607354921-12282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:07:37.989552 10672 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:07:37.989774 10672 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:07:37.989799 10672 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:07:37.990962 10672 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:07:37.990999 10672 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:07:37.991024 10672 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.254.130
I20260812 21:07:37.992509 10672 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.254.148:34715
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568607354921-12282-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568607354921-12282-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.130.keytab
--principal=kudu/127.11.254.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.254.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568607354921-12282-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.254.130
--webserver_port=0
--tserver_master_addrs=127.11.254.190:44439
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10672
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.254.130
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568607354921-12282-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:07:37.992736 10672 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:07:37.992935 10672 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:07:37.998044 10682 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:07:37.998107 10686 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:07:37.998157 10683 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:07:37.998293 10672 server_base.cc:1034] running on GCE node
I20260812 21:07:37.998531 10672 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:07:37.998771 10672 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:07:37.999946 10672 hybrid_clock.cc:648] HybridClock initialized: now 1786568857999928 us; error 32 us; skew 500 ppm
Aug 12 21:07:37 dist-test-slave-vpvm krb5kdc[10264](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568857, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:07:38.002569 10672 init.cc:377] Logged in from keytab as kudu/127.11.254.130@KRBTEST.COM (short username kudu)
I20260812 21:07:38.003042 10672 webserver.cc:460] Webserver started at http://127.11.254.130:44971/ using document root <none> and password file <none>
I20260812 21:07:38.003305 10672 fs_manager.cc:359] Metadata directory not provided
I20260812 21:07:38.003352 10672 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:07:38.003476 10672 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:07:38.004570 10672 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568607354921-12282-0/minicluster-data/ts-1/data/instance:
uuid: "9392e118383d4a9d949d8bd04a81080a"
format_stamp: "Formatted at 2026-08-12 21:07:37 on dist-test-slave-vpvm"
server_key: "a9ac6e62a11393c32e093288beec72ef"
server_key_iv: "7e9df8b4ad6e69d86a89bd59b4a3d14f"
server_key_version: "encryptionkey@0"
I20260812 21:07:38.004933 10672 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568607354921-12282-0/minicluster-data/ts-1/wal/instance:
uuid: "9392e118383d4a9d949d8bd04a81080a"
format_stamp: "Formatted at 2026-08-12 21:07:37 on dist-test-slave-vpvm"
server_key: "a9ac6e62a11393c32e093288beec72ef"
server_key_iv: "7e9df8b4ad6e69d86a89bd59b4a3d14f"
server_key_version: "encryptionkey@0"
I20260812 21:07:38.006305 10672 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260812 21:07:38.007056 10697 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:07:38.007234 10672 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 21:07:38.007318 10672 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568607354921-12282-0/minicluster-data/ts-1/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568607354921-12282-0/minicluster-data/ts-1/wal
uuid: "9392e118383d4a9d949d8bd04a81080a"
format_stamp: "Formatted at 2026-08-12 21:07:37 on dist-test-slave-vpvm"
server_key: "a9ac6e62a11393c32e093288beec72ef"
server_key_iv: "7e9df8b4ad6e69d86a89bd59b4a3d14f"
server_key_version: "encryptionkey@0"
I20260812 21:07:38.007387 10672 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568607354921-12282-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568607354921-12282-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568607354921-12282-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:07:38.014591 10672 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:07:38.016366 10672 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:07:38.016494 10672 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:07:38.016716 10672 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:07:38.017005 10672 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:07:38.017038 10672 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:07:38.017068 10672 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:07:38.017091 10672 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:07:38.022927 10672 rpc_server.cc:307] RPC server started. Bound to: 127.11.254.130:35705
I20260812 21:07:38.022977 10885 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.254.130:35705 every 8 connection(s)
I20260812 21:07:38.023313 10672 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568607354921-12282-0/minicluster-data/ts-1/data/info.pb
Aug 12 21:07:38 dist-test-slave-vpvm krb5kdc[10264](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568857, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.130@KRBTEST.COM for kudu/127.11.254.190@KRBTEST.COM
I20260812 21:07:38.032191 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 10672
I20260812 21:07:38.032299 12282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568607354921-12282-0/minicluster-data/ts-1/wal/instance
I20260812 21:07:38.032505 12282 external_mini_cluster.cc:1428] Setting key 838644488b39b9e9042318a294c658c5
I20260812 21:07:38.032979 10412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:38.024138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.130:36691 (local address 127.11.254.190:44439)
0812 21:07:38.024218 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0812 21:07:38.024221 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 21:07:38.024382 (+ 161us) server_negotiation.cc:407] Connection header received
0812 21:07:38.024971 (+ 589us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:07:38.024973 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:07:38.024987 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:07:38.025041 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:07:38.026042 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:38.026579 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:38.027615 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:38.027824 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:38.027883 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:07:38.030285 (+ 2402us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:07:38.030287 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:07:38.030287 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:07:38.030296 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:07:38.031543 (+ 1247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:07:38.031933 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:07:38.031934 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:07:38.031935 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:07:38.031997 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:07:38.032230 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:07:38.032231 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:07:38.032232 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:07:38.032371 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:07:38.032435 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0812 21:07:38.032785 (+ 350us) server_negotiation.cc:299] Negotiation successful
0812 21:07:38.032888 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260812 21:07:38.033152 10889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:38.024065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:44439 (local address 127.11.254.130:36691)
0812 21:07:38.024253 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0812 21:07:38.024293 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 21:07:38.024862 (+ 569us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:07:38.025115 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:07:38.025119 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:07:38.025713 (+ 594us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:07:38.025920 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:07:38.025925 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:07:38.026676 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:07:38.026677 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:07:38.027521 (+ 844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:07:38.027524 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:07:38.027892 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:07:38.027893 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:07:38.028003 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:07:38.028519 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:07:38.028528 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:07:38.030185 (+ 1657us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:07:38.031626 (+ 1441us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:07:38.031635 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:07:38.031637 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:07:38.031862 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:07:38.032057 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:07:38.032057 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:07:38.032058 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:07:38.032170 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:07:38.032432 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:07:38.032441 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:07:38.032651 (+ 210us) client_negotiation.cc:769] Sending connection context
0812 21:07:38.032753 (+ 102us) client_negotiation.cc:240] Negotiation successful
0812 21:07:38.032923 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":89,"threads_started":1}
I20260812 21:07:38.033617 10887 heartbeater.cc:347] Connected to a master server at 127.11.254.190:44439
I20260812 21:07:38.033723 10887 heartbeater.cc:464] Registering TS with master...
I20260812 21:07:38.033926 10887 heartbeater.cc:511] Master 127.11.254.190:44439 requested a full tablet report, sending...
I20260812 21:07:38.034350 10350 ts_manager.cc:194] Registered new tserver with Master: 9392e118383d4a9d949d8bd04a81080a (127.11.254.130:35705)
I20260812 21:07:38.034799 10350 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.254.130@KRBTEST.COM'} at 127.11.254.130:36691
WARNING: no policy specified for kudu/127.11.254.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.254.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.254.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.131.keytab.
Entry for principal kudu/127.11.254.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.131.keytab.
I20260812 21:07:38.061486 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568607354921-12282-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568607354921-12282-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568607354921-12282-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568607354921-12282-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.254.131:0
--local_ip_for_outbound_sockets=127.11.254.131
--webserver_interface=127.11.254.131
--webserver_port=0
--tserver_master_addrs=127.11.254.190:44439
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.131.keytab
--principal=kudu/127.11.254.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.254.148:34715
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568607354921-12282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568607354921-12282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568607354921-12282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:07:38.146703 10895 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:07:38.146945 10895 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:07:38.146971 10895 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:07:38.148216 10895 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:07:38.148268 10895 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:07:38.148296 10895 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.254.131
I20260812 21:07:38.149889 10895 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.254.148:34715
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568607354921-12282-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568607354921-12282-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.131.keytab
--principal=kudu/127.11.254.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.254.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568607354921-12282-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.254.131
--webserver_port=0
--tserver_master_addrs=127.11.254.190:44439
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10895
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.254.131
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568607354921-12282-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:07:38.150163 10895 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:07:38.150379 10895 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:07:38.155856 10905 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:07:38.155875 10907 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:07:38.155856 10904 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:07:38.156019 10895 server_base.cc:1034] running on GCE node
I20260812 21:07:38.156298 10895 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:07:38.156594 10895 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:07:38.157754 10895 hybrid_clock.cc:648] HybridClock initialized: now 1786568858157719 us; error 52 us; skew 500 ppm
Aug 12 21:07:38 dist-test-slave-vpvm krb5kdc[10264](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568858, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:07:38.160298 10895 init.cc:377] Logged in from keytab as kudu/127.11.254.131@KRBTEST.COM (short username kudu)
I20260812 21:07:38.160787 10895 webserver.cc:460] Webserver started at http://127.11.254.131:36641/ using document root <none> and password file <none>
I20260812 21:07:38.160988 10895 fs_manager.cc:359] Metadata directory not provided
I20260812 21:07:38.161031 10895 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:07:38.161134 10895 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:07:38.162272 10895 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568607354921-12282-0/minicluster-data/ts-2/data/instance:
uuid: "21c8a00217164b1a9a5e4940a81d94dc"
format_stamp: "Formatted at 2026-08-12 21:07:38 on dist-test-slave-vpvm"
server_key: "fe14adaff48879696c759f321cdad4be"
server_key_iv: "e22382f55078ed0ca21d4e30f815949b"
server_key_version: "encryptionkey@0"
I20260812 21:07:38.162609 10895 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568607354921-12282-0/minicluster-data/ts-2/wal/instance:
uuid: "21c8a00217164b1a9a5e4940a81d94dc"
format_stamp: "Formatted at 2026-08-12 21:07:38 on dist-test-slave-vpvm"
server_key: "fe14adaff48879696c759f321cdad4be"
server_key_iv: "e22382f55078ed0ca21d4e30f815949b"
server_key_version: "encryptionkey@0"
I20260812 21:07:38.163964 10895 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 21:07:38.164793 10916 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:07:38.164976 10895 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 21:07:38.165073 10895 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568607354921-12282-0/minicluster-data/ts-2/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568607354921-12282-0/minicluster-data/ts-2/wal
uuid: "21c8a00217164b1a9a5e4940a81d94dc"
format_stamp: "Formatted at 2026-08-12 21:07:38 on dist-test-slave-vpvm"
server_key: "fe14adaff48879696c759f321cdad4be"
server_key_iv: "e22382f55078ed0ca21d4e30f815949b"
server_key_version: "encryptionkey@0"
I20260812 21:07:38.165138 10895 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568607354921-12282-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568607354921-12282-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568607354921-12282-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:07:38.178298 10895 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:07:38.180408 10895 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:07:38.180581 10895 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:07:38.180797 10895 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:07:38.181080 10895 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:07:38.181114 10895 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:07:38.181135 10895 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:07:38.181151 10895 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:07:38.187225 10895 rpc_server.cc:307] RPC server started. Bound to: 127.11.254.131:41207
I20260812 21:07:38.187285 11111 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.254.131:41207 every 8 connection(s)
I20260812 21:07:38.187628 10895 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568607354921-12282-0/minicluster-data/ts-2/data/info.pb
Aug 12 21:07:38 dist-test-slave-vpvm krb5kdc[10264](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568858, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.131@KRBTEST.COM for kudu/127.11.254.190@KRBTEST.COM
I20260812 21:07:38.197161 10412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:38.188326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.131:56815 (local address 127.11.254.190:44439)
0812 21:07:38.188403 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0812 21:07:38.188405 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:07:38.188716 (+ 311us) server_negotiation.cc:407] Connection header received
0812 21:07:38.189221 (+ 505us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:07:38.189222 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:07:38.189233 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:07:38.189283 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:07:38.190257 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:38.190765 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:38.191777 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:38.192004 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:38.192070 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:07:38.194415 (+ 2345us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:07:38.194417 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:07:38.194418 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:07:38.194428 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:07:38.195790 (+ 1362us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:07:38.196223 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:07:38.196225 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:07:38.196225 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:07:38.196282 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:07:38.196506 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:07:38.196507 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:07:38.196507 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:07:38.196635 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:07:38.196692 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0812 21:07:38.196964 (+ 272us) server_negotiation.cc:299] Negotiation successful
0812 21:07:38.197064 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260812 21:07:38.197340 11118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:38.188303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:44439 (local address 127.11.254.131:56815)
0812 21:07:38.188519 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0812 21:07:38.188603 (+ 84us) client_negotiation.cc:174] Beginning negotiation
0812 21:07:38.189138 (+ 535us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:07:38.189343 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:07:38.189346 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:07:38.189970 (+ 624us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:07:38.190181 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:07:38.190187 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:07:38.190851 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:07:38.190852 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:07:38.191681 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:07:38.191683 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:07:38.192110 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:07:38.192111 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:07:38.192236 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:07:38.192739 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:07:38.192749 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:07:38.194273 (+ 1524us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:07:38.195885 (+ 1612us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:07:38.195893 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:07:38.195896 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:07:38.196148 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:07:38.196340 (+ 192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:07:38.196340 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:07:38.196341 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:07:38.196446 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:07:38.196690 (+ 244us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:07:38.196697 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:07:38.196842 (+ 145us) client_negotiation.cc:769] Sending connection context
0812 21:07:38.196931 (+ 89us) client_negotiation.cc:240] Negotiation successful
0812 21:07:38.197115 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":87,"threads_started":1}
I20260812 21:07:38.197636 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 10895
I20260812 21:07:38.197731 12282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568607354921-12282-0/minicluster-data/ts-2/wal/instance
I20260812 21:07:38.197937 12282 external_mini_cluster.cc:1428] Setting key d43e8785dea25343465fb51836f0fe94
I20260812 21:07:38.197924 11112 heartbeater.cc:347] Connected to a master server at 127.11.254.190:44439
I20260812 21:07:38.198081 11112 heartbeater.cc:464] Registering TS with master...
I20260812 21:07:38.198302 11112 heartbeater.cc:511] Master 127.11.254.190:44439 requested a full tablet report, sending...
I20260812 21:07:38.198738 10349 ts_manager.cc:194] Registered new tserver with Master: 21c8a00217164b1a9a5e4940a81d94dc (127.11.254.131:41207)
I20260812 21:07:38.199169 10349 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.254.131@KRBTEST.COM'} at 127.11.254.131:56815
I20260812 21:07:38.210203 12282 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 21:07:38.217942 10412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:38.211627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54082 (local address 127.11.254.190:44439)
0812 21:07:38.211701 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0812 21:07:38.211704 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 21:07:38.211754 (+ 50us) server_negotiation.cc:407] Connection header received
0812 21:07:38.211833 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:07:38.211834 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:07:38.211845 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:07:38.211895 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:07:38.212485 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:38.212984 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:38.213733 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:38.213928 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:38.214018 (+ 90us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:07:38.215344 (+ 1326us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:07:38.215346 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:07:38.215346 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:07:38.215354 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:07:38.216670 (+ 1316us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:07:38.217065 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:07:38.217066 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:07:38.217067 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:07:38.217122 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:07:38.217314 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:07:38.217315 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:07:38.217315 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:07:38.217465 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:07:38.217569 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0812 21:07:38.217771 (+ 202us) server_negotiation.cc:299] Negotiation successful
0812 21:07:38.217839 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 21:07:38.219667 10349 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54082:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 21:07:38.221253 10349 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 21:07:38.228174 11130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:38.223639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.131:41207 (local address 127.0.0.1:33014)
0812 21:07:38.223925 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0812 21:07:38.223973 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 21:07:38.224055 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:07:38.224425 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:07:38.224426 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:07:38.224427 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:07:38.224459 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:07:38.224460 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:07:38.225846 (+ 1386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:07:38.225847 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:07:38.226988 (+ 1141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:07:38.226991 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:07:38.227857 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:07:38.227858 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:07:38.227927 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:07:38.227939 (+ 12us) client_negotiation.cc:769] Sending connection context
0812 21:07:38.228025 (+ 86us) client_negotiation.cc:240] Negotiation successful
0812 21:07:38.228088 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"mutex_wait_us":95,"thread_start_us":56,"threads_started":1}
I20260812 21:07:38.228183 11138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:38.223893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33014 (local address 127.11.254.131:41207)
0812 21:07:38.224147 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0812 21:07:38.224151 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 21:07:38.224162 (+ 11us) server_negotiation.cc:407] Connection header received
0812 21:07:38.224207 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:07:38.224210 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:07:38.224227 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:07:38.224316 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:07:38.224537 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:38.225745 (+ 1208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:38.227075 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:38.227755 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:38.227836 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:07:38.227893 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0812 21:07:38.228044 (+ 151us) server_negotiation.cc:299] Negotiation successful
0812 21:07:38.228089 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":218,"thread_start_us":103,"threads_started":1}
I20260812 21:07:38.228482 11140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:38.223917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58008 (local address 127.11.254.130:35705)
0812 21:07:38.224119 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0812 21:07:38.224125 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 21:07:38.224162 (+ 37us) server_negotiation.cc:407] Connection header received
0812 21:07:38.224207 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:07:38.224209 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:07:38.224227 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:07:38.224317 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:07:38.224594 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:38.225733 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:38.227268 (+ 1535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:38.227957 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:38.228029 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:07:38.228084 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0812 21:07:38.228346 (+ 262us) server_negotiation.cc:299] Negotiation successful
0812 21:07:38.228391 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":82,"threads_started":1}
I20260812 21:07:38.228271 11131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:38.223640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.130:35705 (local address 127.0.0.1:58008)
0812 21:07:38.224055 (+ 415us) negotiation.cc:107] Waiting for socket to connect
0812 21:07:38.224065 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 21:07:38.224110 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:07:38.224333 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:07:38.224337 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:07:38.224338 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:07:38.224423 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:07:38.224425 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:07:38.225866 (+ 1441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:07:38.225867 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:07:38.227175 (+ 1308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:07:38.227179 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:07:38.228043 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:07:38.228044 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:07:38.228113 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:07:38.228116 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 21:07:38.228167 (+ 51us) client_negotiation.cc:240] Negotiation successful
0812 21:07:38.228216 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"mutex_wait_us":30,"thread_start_us":122,"threads_started":1}
I20260812 21:07:38.229240 11132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:38.223646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.129:34871 (local address 127.0.0.1:37224)
0812 21:07:38.224176 (+ 530us) negotiation.cc:107] Waiting for socket to connect
0812 21:07:38.224183 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0812 21:07:38.224212 (+ 29us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:07:38.224721 (+ 509us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:07:38.224722 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:07:38.224724 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:07:38.224775 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:07:38.224777 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:07:38.225797 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:07:38.225798 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:07:38.227193 (+ 1395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:07:38.227195 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:07:38.229006 (+ 1811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:07:38.229008 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:07:38.229068 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:07:38.229073 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 21:07:38.229128 (+ 55us) client_negotiation.cc:240] Negotiation successful
0812 21:07:38.229172 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":515,"mutex_wait_us":88,"thread_start_us":124,"threads_started":1}
I20260812 21:07:38.229270 11135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:38.223743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37224 (local address 127.11.254.129:34871)
0812 21:07:38.224476 (+ 733us) server_negotiation.cc:206] Beginning negotiation
0812 21:07:38.224480 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 21:07:38.224491 (+ 11us) server_negotiation.cc:407] Connection header received
0812 21:07:38.224530 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:07:38.224532 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:07:38.224548 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:07:38.224627 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:07:38.224853 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:38.225692 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:38.228125 (+ 2433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:38.228743 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:38.228819 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:07:38.228878 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0812 21:07:38.229149 (+ 271us) server_negotiation.cc:299] Negotiation successful
0812 21:07:38.229192 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":691,"thread_start_us":90,"threads_started":1}
I20260812 21:07:38.229477 10779 tablet_service.cc:1467] Processing CreateTablet for tablet 351e692ae4ff417fbb6a2afa0e0628a6 (DEFAULT_TABLE table=test-table [id=277f858c381b4a91b76cffdf0d9f3e3b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:07:38.229835 10779 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 351e692ae4ff417fbb6a2afa0e0628a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:07:38.232479 11143 tablet_bootstrap.cc:492] T 351e692ae4ff417fbb6a2afa0e0628a6 P 9392e118383d4a9d949d8bd04a81080a: Bootstrap starting.
I20260812 21:07:38.232770 11003 tablet_service.cc:1467] Processing CreateTablet for tablet 351e692ae4ff417fbb6a2afa0e0628a6 (DEFAULT_TABLE table=test-table [id=277f858c381b4a91b76cffdf0d9f3e3b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:07:38.233078 11003 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 351e692ae4ff417fbb6a2afa0e0628a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:07:38.233145 11143 tablet_bootstrap.cc:654] T 351e692ae4ff417fbb6a2afa0e0628a6 P 9392e118383d4a9d949d8bd04a81080a: Neither blocks nor log segments found. Creating new log.
I20260812 21:07:38.233657 11143 log.cc:824] T 351e692ae4ff417fbb6a2afa0e0628a6 P 9392e118383d4a9d949d8bd04a81080a: Log is configured to *not* fsync() on all Append() calls
I20260812 21:07:38.234637 11143 tablet_bootstrap.cc:492] T 351e692ae4ff417fbb6a2afa0e0628a6 P 9392e118383d4a9d949d8bd04a81080a: No bootstrap required, opened a new log
I20260812 21:07:38.234704 11143 ts_tablet_manager.cc:1397] T 351e692ae4ff417fbb6a2afa0e0628a6 P 9392e118383d4a9d949d8bd04a81080a: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 21:07:38.234730 10552 tablet_service.cc:1467] Processing CreateTablet for tablet 351e692ae4ff417fbb6a2afa0e0628a6 (DEFAULT_TABLE table=test-table [id=277f858c381b4a91b76cffdf0d9f3e3b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:07:38.235083 10552 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 351e692ae4ff417fbb6a2afa0e0628a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:07:38.236121 11152 tablet_bootstrap.cc:492] T 351e692ae4ff417fbb6a2afa0e0628a6 P 21c8a00217164b1a9a5e4940a81d94dc: Bootstrap starting.
I20260812 21:07:38.236188 11143 raft_consensus.cc:348] T 351e692ae4ff417fbb6a2afa0e0628a6 P 9392e118383d4a9d949d8bd04a81080a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8a00217164b1a9a5e4940a81d94dc" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 41207 } } peers { permanent_uuid: "790b4220d8394f438938b2236dcafb61" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 34871 } } peers { permanent_uuid: "9392e118383d4a9d949d8bd04a81080a" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 35705 } }
I20260812 21:07:38.236395 11143 raft_consensus.cc:374] T 351e692ae4ff417fbb6a2afa0e0628a6 P 9392e118383d4a9d949d8bd04a81080a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:07:38.236443 11143 raft_consensus.cc:729] T 351e692ae4ff417fbb6a2afa0e0628a6 P 9392e118383d4a9d949d8bd04a81080a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9392e118383d4a9d949d8bd04a81080a, State: Initialized, Role: FOLLOWER
I20260812 21:07:38.236578 11143 consensus_queue.cc:260] T 351e692ae4ff417fbb6a2afa0e0628a6 P 9392e118383d4a9d949d8bd04a81080a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8a00217164b1a9a5e4940a81d94dc" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 41207 } } peers { permanent_uuid: "790b4220d8394f438938b2236dcafb61" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 34871 } } peers { permanent_uuid: "9392e118383d4a9d949d8bd04a81080a" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 35705 } }
I20260812 21:07:38.236865 11143 ts_tablet_manager.cc:1428] T 351e692ae4ff417fbb6a2afa0e0628a6 P 9392e118383d4a9d949d8bd04a81080a: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260812 21:07:38.236886 11152 tablet_bootstrap.cc:654] T 351e692ae4ff417fbb6a2afa0e0628a6 P 21c8a00217164b1a9a5e4940a81d94dc: Neither blocks nor log segments found. Creating new log.
I20260812 21:07:38.236996 10887 heartbeater.cc:503] Master 127.11.254.190:44439 was elected leader, sending a full tablet report...
I20260812 21:07:38.237413 11152 log.cc:824] T 351e692ae4ff417fbb6a2afa0e0628a6 P 21c8a00217164b1a9a5e4940a81d94dc: Log is configured to *not* fsync() on all Append() calls
I20260812 21:07:38.238173 11156 tablet_bootstrap.cc:492] T 351e692ae4ff417fbb6a2afa0e0628a6 P 790b4220d8394f438938b2236dcafb61: Bootstrap starting.
I20260812 21:07:38.238590 11152 tablet_bootstrap.cc:492] T 351e692ae4ff417fbb6a2afa0e0628a6 P 21c8a00217164b1a9a5e4940a81d94dc: No bootstrap required, opened a new log
I20260812 21:07:38.238687 11152 ts_tablet_manager.cc:1397] T 351e692ae4ff417fbb6a2afa0e0628a6 P 21c8a00217164b1a9a5e4940a81d94dc: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 21:07:38.238884 11156 tablet_bootstrap.cc:654] T 351e692ae4ff417fbb6a2afa0e0628a6 P 790b4220d8394f438938b2236dcafb61: Neither blocks nor log segments found. Creating new log.
I20260812 21:07:38.239447 11156 log.cc:824] T 351e692ae4ff417fbb6a2afa0e0628a6 P 790b4220d8394f438938b2236dcafb61: Log is configured to *not* fsync() on all Append() calls
I20260812 21:07:38.240187 11156 tablet_bootstrap.cc:492] T 351e692ae4ff417fbb6a2afa0e0628a6 P 790b4220d8394f438938b2236dcafb61: No bootstrap required, opened a new log
I20260812 21:07:38.240258 11156 ts_tablet_manager.cc:1397] T 351e692ae4ff417fbb6a2afa0e0628a6 P 790b4220d8394f438938b2236dcafb61: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 21:07:38.240619 11152 raft_consensus.cc:348] T 351e692ae4ff417fbb6a2afa0e0628a6 P 21c8a00217164b1a9a5e4940a81d94dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8a00217164b1a9a5e4940a81d94dc" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 41207 } } peers { permanent_uuid: "790b4220d8394f438938b2236dcafb61" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 34871 } } peers { permanent_uuid: "9392e118383d4a9d949d8bd04a81080a" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 35705 } }
I20260812 21:07:38.240803 11152 raft_consensus.cc:374] T 351e692ae4ff417fbb6a2afa0e0628a6 P 21c8a00217164b1a9a5e4940a81d94dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:07:38.240844 11152 raft_consensus.cc:729] T 351e692ae4ff417fbb6a2afa0e0628a6 P 21c8a00217164b1a9a5e4940a81d94dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21c8a00217164b1a9a5e4940a81d94dc, State: Initialized, Role: FOLLOWER
I20260812 21:07:38.240957 11152 consensus_queue.cc:260] T 351e692ae4ff417fbb6a2afa0e0628a6 P 21c8a00217164b1a9a5e4940a81d94dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8a00217164b1a9a5e4940a81d94dc" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 41207 } } peers { permanent_uuid: "790b4220d8394f438938b2236dcafb61" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 34871 } } peers { permanent_uuid: "9392e118383d4a9d949d8bd04a81080a" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 35705 } }
I20260812 21:07:38.241266 11152 ts_tablet_manager.cc:1428] T 351e692ae4ff417fbb6a2afa0e0628a6 P 21c8a00217164b1a9a5e4940a81d94dc: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 21:07:38.241293 11112 heartbeater.cc:503] Master 127.11.254.190:44439 was elected leader, sending a full tablet report...
I20260812 21:07:38.242051 11156 raft_consensus.cc:348] T 351e692ae4ff417fbb6a2afa0e0628a6 P 790b4220d8394f438938b2236dcafb61 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8a00217164b1a9a5e4940a81d94dc" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 41207 } } peers { permanent_uuid: "790b4220d8394f438938b2236dcafb61" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 34871 } } peers { permanent_uuid: "9392e118383d4a9d949d8bd04a81080a" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 35705 } }
I20260812 21:07:38.242219 11156 raft_consensus.cc:374] T 351e692ae4ff417fbb6a2afa0e0628a6 P 790b4220d8394f438938b2236dcafb61 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:07:38.242269 11156 raft_consensus.cc:729] T 351e692ae4ff417fbb6a2afa0e0628a6 P 790b4220d8394f438938b2236dcafb61 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 790b4220d8394f438938b2236dcafb61, State: Initialized, Role: FOLLOWER
I20260812 21:07:38.242377 11156 consensus_queue.cc:260] T 351e692ae4ff417fbb6a2afa0e0628a6 P 790b4220d8394f438938b2236dcafb61 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8a00217164b1a9a5e4940a81d94dc" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 41207 } } peers { permanent_uuid: "790b4220d8394f438938b2236dcafb61" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 34871 } } peers { permanent_uuid: "9392e118383d4a9d949d8bd04a81080a" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 35705 } }
I20260812 21:07:38.242633 11156 ts_tablet_manager.cc:1428] T 351e692ae4ff417fbb6a2afa0e0628a6 P 790b4220d8394f438938b2236dcafb61: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260812 21:07:38.242704 10663 heartbeater.cc:503] Master 127.11.254.190:44439 was elected leader, sending a full tablet report...
W20260812 21:07:38.273921 10888 tablet.cc:2369] T 351e692ae4ff417fbb6a2afa0e0628a6 P 9392e118383d4a9d949d8bd04a81080a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 21:07:38.367239 10664 tablet.cc:2369] T 351e692ae4ff417fbb6a2afa0e0628a6 P 790b4220d8394f438938b2236dcafb61: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 21:07:38.368506 11164 raft_consensus.cc:482] T 351e692ae4ff417fbb6a2afa0e0628a6 P 21c8a00217164b1a9a5e4940a81d94dc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 21:07:38.368647 11164 raft_consensus.cc:504] T 351e692ae4ff417fbb6a2afa0e0628a6 P 21c8a00217164b1a9a5e4940a81d94dc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8a00217164b1a9a5e4940a81d94dc" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 41207 } } peers { permanent_uuid: "790b4220d8394f438938b2236dcafb61" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 34871 } } peers { permanent_uuid: "9392e118383d4a9d949d8bd04a81080a" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 35705 } }
I20260812 21:07:38.369000 11164 leader_election.cc:288] T 351e692ae4ff417fbb6a2afa0e0628a6 P 21c8a00217164b1a9a5e4940a81d94dc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 790b4220d8394f438938b2236dcafb61 (127.11.254.129:34871), 9392e118383d4a9d949d8bd04a81080a (127.11.254.130:35705)
I20260812 21:07:38.371724 11118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:38.369176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.130:35705 (local address 127.11.254.131:41561)
0812 21:07:38.369258 (+ 82us) negotiation.cc:107] Waiting for socket to connect
0812 21:07:38.369270 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 21:07:38.369321 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:07:38.369444 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:07:38.369444 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:07:38.369446 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:07:38.369522 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:07:38.369524 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:07:38.370198 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:07:38.370199 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:07:38.370937 (+ 738us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:07:38.370939 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:07:38.371472 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:07:38.371473 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:07:38.371558 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:07:38.371561 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 21:07:38.371605 (+ 44us) client_negotiation.cc:240] Negotiation successful
0812 21:07:38.371650 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60,"mutex_wait_us":1}
I20260812 21:07:38.371722 11140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:38.369292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.131:41561 (local address 127.11.254.130:35705)
0812 21:07:38.369337 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0812 21:07:38.369339 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:07:38.369348 (+ 9us) server_negotiation.cc:407] Connection header received
0812 21:07:38.369378 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:07:38.369379 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:07:38.369387 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:07:38.369434 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:07:38.369592 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:38.370130 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:38.371028 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:38.371405 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:38.371467 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:07:38.371476 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 21:07:38.371622 (+ 146us) server_negotiation.cc:299] Negotiation successful
0812 21:07:38.371658 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 21:07:38.371881 11135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:38.369275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.131:58649 (local address 127.11.254.129:34871)
0812 21:07:38.369323 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0812 21:07:38.369325 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:07:38.369471 (+ 146us) server_negotiation.cc:407] Connection header received
0812 21:07:38.369531 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:07:38.369532 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:07:38.369541 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:07:38.369591 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:07:38.369738 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:38.370257 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:38.371179 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:38.371543 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:38.371603 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:07:38.371611 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 21:07:38.371766 (+ 155us) server_negotiation.cc:299] Negotiation successful
0812 21:07:38.371805 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 21:07:38.372074 11167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:38.369199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.129:34871 (local address 127.11.254.131:58649)
0812 21:07:38.369391 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0812 21:07:38.369400 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 21:07:38.369463 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:07:38.369604 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:07:38.369604 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:07:38.369606 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:07:38.369676 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:07:38.369678 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:07:38.370330 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:07:38.370331 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:07:38.371094 (+ 763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:07:38.371096 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:07:38.371624 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:07:38.371625 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:07:38.371704 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:07:38.371707 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 21:07:38.371752 (+ 45us) client_negotiation.cc:240] Negotiation successful
0812 21:07:38.371911 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":115,"threads_started":1}
I20260812 21:07:38.372058 10808 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "351e692ae4ff417fbb6a2afa0e0628a6" candidate_uuid: "21c8a00217164b1a9a5e4940a81d94dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9392e118383d4a9d949d8bd04a81080a" is_pre_election: true
I20260812 21:07:38.372231 10808 raft_consensus.cc:2454] T 351e692ae4ff417fbb6a2afa0e0628a6 P 9392e118383d4a9d949d8bd04a81080a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 21c8a00217164b1a9a5e4940a81d94dc in term 0.
I20260812 21:07:38.372407 10588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "351e692ae4ff417fbb6a2afa0e0628a6" candidate_uuid: "21c8a00217164b1a9a5e4940a81d94dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "790b4220d8394f438938b2236dcafb61" is_pre_election: true
I20260812 21:07:38.372478 10919 leader_election.cc:302] T 351e692ae4ff417fbb6a2afa0e0628a6 P 21c8a00217164b1a9a5e4940a81d94dc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21c8a00217164b1a9a5e4940a81d94dc, 9392e118383d4a9d949d8bd04a81080a; no voters:
I20260812 21:07:38.372555 10588 raft_consensus.cc:2454] T 351e692ae4ff417fbb6a2afa0e0628a6 P 790b4220d8394f438938b2236dcafb61 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 21c8a00217164b1a9a5e4940a81d94dc in term 0.
I20260812 21:07:38.372622 11164 raft_consensus.cc:2781] T 351e692ae4ff417fbb6a2afa0e0628a6 P 21c8a00217164b1a9a5e4940a81d94dc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 21:07:38.372701 11164 raft_consensus.cc:482] T 351e692ae4ff417fbb6a2afa0e0628a6 P 21c8a00217164b1a9a5e4940a81d94dc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 21:07:38.372737 11164 raft_consensus.cc:3037] T 351e692ae4ff417fbb6a2afa0e0628a6 P 21c8a00217164b1a9a5e4940a81d94dc [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:07:38.373450 11164 raft_consensus.cc:504] T 351e692ae4ff417fbb6a2afa0e0628a6 P 21c8a00217164b1a9a5e4940a81d94dc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8a00217164b1a9a5e4940a81d94dc" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 41207 } } peers { permanent_uuid: "790b4220d8394f438938b2236dcafb61" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 34871 } } peers { permanent_uuid: "9392e118383d4a9d949d8bd04a81080a" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 35705 } }
I20260812 21:07:38.373572 11164 leader_election.cc:288] T 351e692ae4ff417fbb6a2afa0e0628a6 P 21c8a00217164b1a9a5e4940a81d94dc [CANDIDATE]: Term 1 election: Requested vote from peers 790b4220d8394f438938b2236dcafb61 (127.11.254.129:34871), 9392e118383d4a9d949d8bd04a81080a (127.11.254.130:35705)
I20260812 21:07:38.373742 10808 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "351e692ae4ff417fbb6a2afa0e0628a6" candidate_uuid: "21c8a00217164b1a9a5e4940a81d94dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9392e118383d4a9d949d8bd04a81080a"
I20260812 21:07:38.373742 10588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "351e692ae4ff417fbb6a2afa0e0628a6" candidate_uuid: "21c8a00217164b1a9a5e4940a81d94dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "790b4220d8394f438938b2236dcafb61"
I20260812 21:07:38.373826 10808 raft_consensus.cc:3037] T 351e692ae4ff417fbb6a2afa0e0628a6 P 9392e118383d4a9d949d8bd04a81080a [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:07:38.373826 10588 raft_consensus.cc:3037] T 351e692ae4ff417fbb6a2afa0e0628a6 P 790b4220d8394f438938b2236dcafb61 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:07:38.374734 10808 raft_consensus.cc:2454] T 351e692ae4ff417fbb6a2afa0e0628a6 P 9392e118383d4a9d949d8bd04a81080a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 21c8a00217164b1a9a5e4940a81d94dc in term 1.
I20260812 21:07:38.374734 10588 raft_consensus.cc:2454] T 351e692ae4ff417fbb6a2afa0e0628a6 P 790b4220d8394f438938b2236dcafb61 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 21c8a00217164b1a9a5e4940a81d94dc in term 1.
I20260812 21:07:38.374914 10919 leader_election.cc:302] T 351e692ae4ff417fbb6a2afa0e0628a6 P 21c8a00217164b1a9a5e4940a81d94dc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21c8a00217164b1a9a5e4940a81d94dc, 9392e118383d4a9d949d8bd04a81080a; no voters:
I20260812 21:07:38.375031 11164 raft_consensus.cc:2781] T 351e692ae4ff417fbb6a2afa0e0628a6 P 21c8a00217164b1a9a5e4940a81d94dc [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:07:38.375200 11164 raft_consensus.cc:686] T 351e692ae4ff417fbb6a2afa0e0628a6 P 21c8a00217164b1a9a5e4940a81d94dc [term 1 LEADER]: Becoming Leader. State: Replica: 21c8a00217164b1a9a5e4940a81d94dc, State: Running, Role: LEADER
I20260812 21:07:38.375295 11164 consensus_queue.cc:237] T 351e692ae4ff417fbb6a2afa0e0628a6 P 21c8a00217164b1a9a5e4940a81d94dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8a00217164b1a9a5e4940a81d94dc" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 41207 } } peers { permanent_uuid: "790b4220d8394f438938b2236dcafb61" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 34871 } } peers { permanent_uuid: "9392e118383d4a9d949d8bd04a81080a" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 35705 } }
I20260812 21:07:38.376116 10350 catalog_manager.cc:5495] T 351e692ae4ff417fbb6a2afa0e0628a6 P 21c8a00217164b1a9a5e4940a81d94dc reported cstate change: term changed from 0 to 1, leader changed from <none> to 21c8a00217164b1a9a5e4940a81d94dc (127.11.254.131). New cstate: current_term: 1 leader_uuid: "21c8a00217164b1a9a5e4940a81d94dc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8a00217164b1a9a5e4940a81d94dc" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 41207 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "790b4220d8394f438938b2236dcafb61" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 34871 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9392e118383d4a9d949d8bd04a81080a" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 35705 } health_report { overall_health: UNKNOWN } } }
I20260812 21:07:38.416023 11138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:38.413619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33020 (local address 127.11.254.131:41207)
0812 21:07:38.413684 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0812 21:07:38.413686 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:07:38.413694 (+ 8us) server_negotiation.cc:407] Connection header received
0812 21:07:38.413732 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:07:38.413733 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:07:38.413750 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:07:38.413813 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 21:07:38.413981 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:38.414477 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:38.415199 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:38.415445 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:38.415504 (+ 59us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:07:38.415669 (+ 165us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 21:07:38.415769 (+ 100us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 21:07:38.415854 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0812 21:07:38.415925 (+ 71us) server_negotiation.cc:299] Negotiation successful
0812 21:07:38.415935 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 21:07:38.417882 10588 raft_consensus.cc:1261] T 351e692ae4ff417fbb6a2afa0e0628a6 P 790b4220d8394f438938b2236dcafb61 [term 1 FOLLOWER]: Refusing update from remote peer 21c8a00217164b1a9a5e4940a81d94dc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 21:07:38.417883 10808 raft_consensus.cc:1261] T 351e692ae4ff417fbb6a2afa0e0628a6 P 9392e118383d4a9d949d8bd04a81080a [term 1 FOLLOWER]: Refusing update from remote peer 21c8a00217164b1a9a5e4940a81d94dc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 21:07:38.418267 11168 consensus_queue.cc:1035] T 351e692ae4ff417fbb6a2afa0e0628a6 P 21c8a00217164b1a9a5e4940a81d94dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "790b4220d8394f438938b2236dcafb61" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 34871 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 21:07:38.418401 11164 consensus_queue.cc:1035] T 351e692ae4ff417fbb6a2afa0e0628a6 P 21c8a00217164b1a9a5e4940a81d94dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "9392e118383d4a9d949d8bd04a81080a" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 35705 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 21:07:38.420598 11172 mvcc.cc:205] Tried to move back new op lower bound from 7317786044076851200 to 7317786043905822720. Current Snapshot: MvccSnapshot[applied={T|T < 7317786044076851200}]
I20260812 21:07:38.421101 11174 mvcc.cc:205] Tried to move back new op lower bound from 7317786044076851200 to 7317786043905822720. Current Snapshot: MvccSnapshot[applied={T|T < 7317786044076851200}]
I20260812 21:07:38.423183 10349 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54082:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 21:07:38.423272 10349 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54082:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 21:07:38.423964 10349 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ee3e88fa99e040f587bf2da48520d7c1: Sending DeleteTablet for 3 replicas of tablet 351e692ae4ff417fbb6a2afa0e0628a6
I20260812 21:07:38.424274 11003 tablet_service.cc:1514] Processing DeleteTablet for tablet 351e692ae4ff417fbb6a2afa0e0628a6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:07:38 UTC) from {username='kudu', principal='kudu/127.11.254.190@KRBTEST.COM'} at 127.0.0.1:33014
I20260812 21:07:38.424355 10552 tablet_service.cc:1514] Processing DeleteTablet for tablet 351e692ae4ff417fbb6a2afa0e0628a6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:07:38 UTC) from {username='kudu', principal='kudu/127.11.254.190@KRBTEST.COM'} at 127.0.0.1:37224
I20260812 21:07:38.424561 10779 tablet_service.cc:1514] Processing DeleteTablet for tablet 351e692ae4ff417fbb6a2afa0e0628a6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:07:38 UTC) from {username='kudu', principal='kudu/127.11.254.190@KRBTEST.COM'} at 127.0.0.1:58008
I20260812 21:07:38.424693 11188 tablet_replica.cc:330] T 351e692ae4ff417fbb6a2afa0e0628a6 P 21c8a00217164b1a9a5e4940a81d94dc: stopping tablet replica
I20260812 21:07:38.424752 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 10448
I20260812 21:07:38.424796 11188 raft_consensus.cc:2229] T 351e692ae4ff417fbb6a2afa0e0628a6 P 21c8a00217164b1a9a5e4940a81d94dc [term 1 LEADER]: Raft consensus shutting down.
I20260812 21:07:38.424784 11189 tablet_replica.cc:330] T 351e692ae4ff417fbb6a2afa0e0628a6 P 790b4220d8394f438938b2236dcafb61: stopping tablet replica
I20260812 21:07:38.437096 11190 tablet_replica.cc:330] T 351e692ae4ff417fbb6a2afa0e0628a6 P 9392e118383d4a9d949d8bd04a81080a: stopping tablet replica
I20260812 21:07:38.437256 11190 raft_consensus.cc:2229] T 351e692ae4ff417fbb6a2afa0e0628a6 P 9392e118383d4a9d949d8bd04a81080a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 21:07:38.437369 11190 raft_consensus.cc:2258] T 351e692ae4ff417fbb6a2afa0e0628a6 P 9392e118383d4a9d949d8bd04a81080a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 21:07:38.446343 11190 ts_tablet_manager.cc:1905] T 351e692ae4ff417fbb6a2afa0e0628a6 P 9392e118383d4a9d949d8bd04a81080a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 21:07:38.447966 11190 ts_tablet_manager.cc:1918] T 351e692ae4ff417fbb6a2afa0e0628a6 P 9392e118383d4a9d949d8bd04a81080a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 21:07:38.448071 11190 log.cc:1196] T 351e692ae4ff417fbb6a2afa0e0628a6 P 9392e118383d4a9d949d8bd04a81080a: Deleting WAL directory at /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568607354921-12282-0/minicluster-data/ts-1/wal/wals/351e692ae4ff417fbb6a2afa0e0628a6
I20260812 21:07:38.448360 11190 ts_tablet_manager.cc:1939] T 351e692ae4ff417fbb6a2afa0e0628a6 P 9392e118383d4a9d949d8bd04a81080a: Deleting consensus metadata
I20260812 21:07:38.448886 10322 catalog_manager.cc:4841] TS 9392e118383d4a9d949d8bd04a81080a (127.11.254.130:35705): tablet 351e692ae4ff417fbb6a2afa0e0628a6 (table test-table [id=277f858c381b4a91b76cffdf0d9f3e3b]) successfully deleted
I20260812 21:07:38.449074 11188 raft_consensus.cc:2258] T 351e692ae4ff417fbb6a2afa0e0628a6 P 21c8a00217164b1a9a5e4940a81d94dc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 21:07:38.449584 11188 ts_tablet_manager.cc:1905] T 351e692ae4ff417fbb6a2afa0e0628a6 P 21c8a00217164b1a9a5e4940a81d94dc: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260812 21:07:38.450220 10325 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.254.129:34871 (error 108)
I20260812 21:07:38.450507 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 10672
I20260812 21:07:38.450979 11132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:38.450851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.129:34871 (local address 127.0.0.1:37232)
0812 21:07:38.450907 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0812 21:07:38.450941 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.129:34871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260812 21:07:38.451236 11188 ts_tablet_manager.cc:1918] T 351e692ae4ff417fbb6a2afa0e0628a6 P 21c8a00217164b1a9a5e4940a81d94dc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 21:07:38.451289 11188 log.cc:1196] T 351e692ae4ff417fbb6a2afa0e0628a6 P 21c8a00217164b1a9a5e4940a81d94dc: Deleting WAL directory at /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786568607354921-12282-0/minicluster-data/ts-2/wal/wals/351e692ae4ff417fbb6a2afa0e0628a6
I20260812 21:07:38.451495 11188 ts_tablet_manager.cc:1939] T 351e692ae4ff417fbb6a2afa0e0628a6 P 21c8a00217164b1a9a5e4940a81d94dc: Deleting consensus metadata
I20260812 21:07:38.451999 10324 catalog_manager.cc:4841] TS 21c8a00217164b1a9a5e4940a81d94dc (127.11.254.131:41207): tablet 351e692ae4ff417fbb6a2afa0e0628a6 (table test-table [id=277f858c381b4a91b76cffdf0d9f3e3b]) successfully deleted
W20260812 21:07:38.453310 10325 catalog_manager.cc:4584] TS 790b4220d8394f438938b2236dcafb61 (127.11.254.129:34871): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 351e692ae4ff417fbb6a2afa0e0628a6: Network error: Client connection negotiation failed: client connection to 127.11.254.129:34871: connect: Connection refused (error 111)
I20260812 21:07:38.462455 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 10895
I20260812 21:07:38.483211 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 10285
2026-08-12T21:07:38Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2258 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-12T21:07:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T21:07:38Z Disabled control of system clock
I20260812 21:07:38.517927 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786568607354921-12282-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786568607354921-12282-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786568607354921-12282-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.254.190:46625
--webserver_interface=127.11.254.190
--webserver_port=0
--builtin_ntp_servers=127.11.254.148:35813
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.254.190:46625
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 21:07:38.598721 11197 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:07:38.598930 11197 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:07:38.598949 11197 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:07:38.600033 11197 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 21:07:38.600067 11197 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:07:38.600078 11197 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:07:38.600090 11197 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 21:07:38.600100 11197 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 21:07:38.601374 11197 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.254.148:35813
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786568607354921-12282-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786568607354921-12282-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.254.190:46625
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.254.190:46625
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.254.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11197
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786568607354921-12282-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:07:38.601577 11197 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:07:38.601830 11197 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:07:38.606814 11209 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:07:38.606822 11197 server_base.cc:1034] running on GCE node
W20260812 21:07:38.606794 11207 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:07:38.606794 11205 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:07:38.607322 11197 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:07:38.607614 11197 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:07:38.608757 11197 hybrid_clock.cc:648] HybridClock initialized: now 1786568858608738 us; error 31 us; skew 500 ppm
I20260812 21:07:38.610018 11197 webserver.cc:460] Webserver started at http://127.11.254.190:39005/ using document root <none> and password file <none>
I20260812 21:07:38.610267 11197 fs_manager.cc:359] Metadata directory not provided
I20260812 21:07:38.610319 11197 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:07:38.610476 11197 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:07:38.611595 11197 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786568607354921-12282-0/minicluster-data/master-0/data/instance:
uuid: "bec4359db5a54b9fbd8ce26632e9432f"
format_stamp: "Formatted at 2026-08-12 21:07:38 on dist-test-slave-vpvm"
server_key: "5c2bae3b739ae1ca76159a1aeb9be3e7"
server_key_iv: "9cf4cbccc711ceef92e1eaa20219bbb8"
server_key_version: "encryptionkey@0"
I20260812 21:07:38.611933 11197 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786568607354921-12282-0/minicluster-data/master-0/wal/instance:
uuid: "bec4359db5a54b9fbd8ce26632e9432f"
format_stamp: "Formatted at 2026-08-12 21:07:38 on dist-test-slave-vpvm"
server_key: "5c2bae3b739ae1ca76159a1aeb9be3e7"
server_key_iv: "9cf4cbccc711ceef92e1eaa20219bbb8"
server_key_version: "encryptionkey@0"
I20260812 21:07:38.613174 11197 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 21:07:38.613914 11220 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:07:38.614115 11197 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 21:07:38.614202 11197 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786568607354921-12282-0/minicluster-data/master-0/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786568607354921-12282-0/minicluster-data/master-0/wal
uuid: "bec4359db5a54b9fbd8ce26632e9432f"
format_stamp: "Formatted at 2026-08-12 21:07:38 on dist-test-slave-vpvm"
server_key: "5c2bae3b739ae1ca76159a1aeb9be3e7"
server_key_iv: "9cf4cbccc711ceef92e1eaa20219bbb8"
server_key_version: "encryptionkey@0"
I20260812 21:07:38.614264 11197 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786568607354921-12282-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786568607354921-12282-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786568607354921-12282-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:07:38.639629 11197 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:07:38.639974 11197 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:07:38.640126 11197 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:07:38.644093 11197 rpc_server.cc:307] RPC server started. Bound to: 127.11.254.190:46625
I20260812 21:07:38.644134 11306 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.254.190:46625 every 8 connection(s)
I20260812 21:07:38.644462 11197 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
I20260812 21:07:38.645437 11307 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:07:38.647832 11307 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bec4359db5a54b9fbd8ce26632e9432f: Bootstrap starting.
I20260812 21:07:38.648384 11307 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bec4359db5a54b9fbd8ce26632e9432f: Neither blocks nor log segments found. Creating new log.
I20260812 21:07:38.648648 11307 log.cc:824] T 00000000000000000000000000000000 P bec4359db5a54b9fbd8ce26632e9432f: Log is configured to *not* fsync() on all Append() calls
I20260812 21:07:38.649264 11307 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bec4359db5a54b9fbd8ce26632e9432f: No bootstrap required, opened a new log
I20260812 21:07:38.650941 11307 raft_consensus.cc:348] T 00000000000000000000000000000000 P bec4359db5a54b9fbd8ce26632e9432f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bec4359db5a54b9fbd8ce26632e9432f" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 46625 } }
I20260812 21:07:38.651091 11307 raft_consensus.cc:374] T 00000000000000000000000000000000 P bec4359db5a54b9fbd8ce26632e9432f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:07:38.651114 11307 raft_consensus.cc:729] T 00000000000000000000000000000000 P bec4359db5a54b9fbd8ce26632e9432f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bec4359db5a54b9fbd8ce26632e9432f, State: Initialized, Role: FOLLOWER
I20260812 21:07:38.651230 11307 consensus_queue.cc:260] T 00000000000000000000000000000000 P bec4359db5a54b9fbd8ce26632e9432f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bec4359db5a54b9fbd8ce26632e9432f" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 46625 } }
I20260812 21:07:38.651306 11307 raft_consensus.cc:388] T 00000000000000000000000000000000 P bec4359db5a54b9fbd8ce26632e9432f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 21:07:38.651335 11307 raft_consensus.cc:482] T 00000000000000000000000000000000 P bec4359db5a54b9fbd8ce26632e9432f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 21:07:38.651366 11307 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bec4359db5a54b9fbd8ce26632e9432f [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:07:38.651845 11307 raft_consensus.cc:504] T 00000000000000000000000000000000 P bec4359db5a54b9fbd8ce26632e9432f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bec4359db5a54b9fbd8ce26632e9432f" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 46625 } }
I20260812 21:07:38.651930 11307 leader_election.cc:302] T 00000000000000000000000000000000 P bec4359db5a54b9fbd8ce26632e9432f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bec4359db5a54b9fbd8ce26632e9432f; no voters:
I20260812 21:07:38.652063 11307 leader_election.cc:288] T 00000000000000000000000000000000 P bec4359db5a54b9fbd8ce26632e9432f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 21:07:38.652122 11312 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bec4359db5a54b9fbd8ce26632e9432f [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:07:38.652285 11312 raft_consensus.cc:686] T 00000000000000000000000000000000 P bec4359db5a54b9fbd8ce26632e9432f [term 1 LEADER]: Becoming Leader. State: Replica: bec4359db5a54b9fbd8ce26632e9432f, State: Running, Role: LEADER
I20260812 21:07:38.652299 11307 sys_catalog.cc:564] T 00000000000000000000000000000000 P bec4359db5a54b9fbd8ce26632e9432f [sys.catalog]: configured and running, proceeding with master startup.
I20260812 21:07:38.652385 11312 consensus_queue.cc:237] T 00000000000000000000000000000000 P bec4359db5a54b9fbd8ce26632e9432f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bec4359db5a54b9fbd8ce26632e9432f" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 46625 } }
I20260812 21:07:38.652791 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 11197
I20260812 21:07:38.652890 12282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786568607354921-12282-0/minicluster-data/master-0/wal/instance
I20260812 21:07:38.653105 12282 external_mini_cluster.cc:1428] Setting key 7601841159b0cbe05c3fb030c1b1c9cd
I20260812 21:07:38.653887 11316 sys_catalog.cc:455] T 00000000000000000000000000000000 P bec4359db5a54b9fbd8ce26632e9432f [sys.catalog]: SysCatalogTable state changed. Reason: New leader bec4359db5a54b9fbd8ce26632e9432f. Latest consensus state: current_term: 1 leader_uuid: "bec4359db5a54b9fbd8ce26632e9432f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bec4359db5a54b9fbd8ce26632e9432f" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 46625 } } }
I20260812 21:07:38.654035 11316 sys_catalog.cc:458] T 00000000000000000000000000000000 P bec4359db5a54b9fbd8ce26632e9432f [sys.catalog]: This master's current role is: LEADER
I20260812 21:07:38.653915 11314 sys_catalog.cc:455] T 00000000000000000000000000000000 P bec4359db5a54b9fbd8ce26632e9432f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bec4359db5a54b9fbd8ce26632e9432f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bec4359db5a54b9fbd8ce26632e9432f" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 46625 } } }
I20260812 21:07:38.654137 11314 sys_catalog.cc:458] T 00000000000000000000000000000000 P bec4359db5a54b9fbd8ce26632e9432f [sys.catalog]: This master's current role is: LEADER
W20260812 21:07:38.654417 11350 catalog_manager.cc:1558] T 00000000000000000000000000000000 P bec4359db5a54b9fbd8ce26632e9432f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 21:07:38.654459 11350 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 21:07:38.654695 11357 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 21:07:38.654878 11357 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 21:07:38.657528 11357 catalog_manager.cc:1347] Generated new cluster ID: bb8d7c30512b425eac71bce7e3b2509b
I20260812 21:07:38.657585 11357 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 21:07:38.659153 11353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:38.653749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43006 (local address 127.11.254.190:46625)
0812 21:07:38.653976 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0812 21:07:38.653985 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 21:07:38.654021 (+ 36us) server_negotiation.cc:407] Connection header received
0812 21:07:38.654378 (+ 357us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:07:38.654386 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:07:38.654459 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:07:38.654605 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:07:38.654842 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:38.656006 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:38.657088 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:38.657577 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:38.657694 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:07:38.658535 (+ 841us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:07:38.658540 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:07:38.658540 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 21:07:38.658711 (+ 171us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:07:38.658745 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 21:07:38.658756 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:07:38.658814 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0812 21:07:38.658888 (+ 74us) server_negotiation.cc:299] Negotiation successful
0812 21:07:38.658966 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":70,"threads_started":1}
I20260812 21:07:38.662174 11357 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 21:07:38.662695 11357 catalog_manager.cc:1504] Loading token signing keys...
I20260812 21:07:38.670384 11357 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bec4359db5a54b9fbd8ce26632e9432f: Generated new TSK 0
I20260812 21:07:38.670616 11357 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 21:07:38.676323 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786568607354921-12282-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786568607354921-12282-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786568607354921-12282-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786568607354921-12282-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.254.129:0
--local_ip_for_outbound_sockets=127.11.254.129
--webserver_interface=127.11.254.129
--webserver_port=0
--tserver_master_addrs=127.11.254.190:46625
--builtin_ntp_servers=127.11.254.148:35813
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 21:07:38.759886 11362 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:07:38.760103 11362 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:07:38.760126 11362 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:07:38.761276 11362 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:07:38.761319 11362 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:07:38.761344 11362 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.254.129
I20260812 21:07:38.762771 11362 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.254.148:35813
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786568607354921-12282-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786568607354921-12282-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.254.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786568607354921-12282-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.254.129
--webserver_port=0
--tserver_master_addrs=127.11.254.190:46625
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11362
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.254.129
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786568607354921-12282-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:07:38.763021 11362 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:07:38.763348 11362 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:07:38.768337 11372 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:07:38.768337 11367 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:07:38.768321 11369 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:07:38.768769 11362 server_base.cc:1034] running on GCE node
I20260812 21:07:38.768963 11362 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:07:38.769196 11362 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:07:38.770381 11362 hybrid_clock.cc:648] HybridClock initialized: now 1786568858770353 us; error 37 us; skew 500 ppm
I20260812 21:07:38.771723 11362 webserver.cc:460] Webserver started at http://127.11.254.129:36247/ using document root <none> and password file <none>
I20260812 21:07:38.771965 11362 fs_manager.cc:359] Metadata directory not provided
I20260812 21:07:38.772018 11362 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:07:38.772147 11362 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:07:38.773216 11362 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786568607354921-12282-0/minicluster-data/ts-0/data/instance:
uuid: "1cdd0b991e964392ba8e366e0d441585"
format_stamp: "Formatted at 2026-08-12 21:07:38 on dist-test-slave-vpvm"
server_key: "d3444baa6bd8b64724380322ffc81385"
server_key_iv: "e7624e3068c8f10542738bfef4e28e60"
server_key_version: "encryptionkey@0"
I20260812 21:07:38.773571 11362 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786568607354921-12282-0/minicluster-data/ts-0/wal/instance:
uuid: "1cdd0b991e964392ba8e366e0d441585"
format_stamp: "Formatted at 2026-08-12 21:07:38 on dist-test-slave-vpvm"
server_key: "d3444baa6bd8b64724380322ffc81385"
server_key_iv: "e7624e3068c8f10542738bfef4e28e60"
server_key_version: "encryptionkey@0"
I20260812 21:07:38.774958 11362 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 21:07:38.775795 11387 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:07:38.775977 11362 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 21:07:38.776077 11362 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786568607354921-12282-0/minicluster-data/ts-0/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786568607354921-12282-0/minicluster-data/ts-0/wal
uuid: "1cdd0b991e964392ba8e366e0d441585"
format_stamp: "Formatted at 2026-08-12 21:07:38 on dist-test-slave-vpvm"
server_key: "d3444baa6bd8b64724380322ffc81385"
server_key_iv: "e7624e3068c8f10542738bfef4e28e60"
server_key_version: "encryptionkey@0"
I20260812 21:07:38.776163 11362 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786568607354921-12282-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786568607354921-12282-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786568607354921-12282-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:07:38.802155 11362 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:07:38.802459 11362 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:07:38.802611 11362 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:07:38.802831 11362 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:07:38.803200 11362 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:07:38.803238 11362 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:07:38.803259 11362 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:07:38.803272 11362 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:07:38.808766 11362 rpc_server.cc:307] RPC server started. Bound to: 127.11.254.129:40409
I20260812 21:07:38.808836 11597 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.254.129:40409 every 8 connection(s)
I20260812 21:07:38.809149 11362 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786568607354921-12282-0/minicluster-data/ts-0/data/info.pb
I20260812 21:07:38.812355 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 11362
I20260812 21:07:38.812469 12282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786568607354921-12282-0/minicluster-data/ts-0/wal/instance
I20260812 21:07:38.812673 12282 external_mini_cluster.cc:1428] Setting key f96e618041f29c6d0e122908d5e239af
I20260812 21:07:38.813813 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786568607354921-12282-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786568607354921-12282-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786568607354921-12282-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786568607354921-12282-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.254.130:0
--local_ip_for_outbound_sockets=127.11.254.130
--webserver_interface=127.11.254.130
--webserver_port=0
--tserver_master_addrs=127.11.254.190:46625
--builtin_ntp_servers=127.11.254.148:35813
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 21:07:38.814927 11353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:38.809870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.129:47965 (local address 127.11.254.190:46625)
0812 21:07:38.809987 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0812 21:07:38.809990 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 21:07:38.810165 (+ 175us) server_negotiation.cc:407] Connection header received
0812 21:07:38.810731 (+ 566us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:07:38.810732 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:07:38.810745 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:07:38.810812 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:07:38.811522 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:38.812068 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:38.813424 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:38.813696 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:38.813770 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:07:38.814677 (+ 907us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:07:38.814678 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:07:38.814679 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 21:07:38.814683 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:07:38.814701 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 21:07:38.814706 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:07:38.814759 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0812 21:07:38.814817 (+ 58us) server_negotiation.cc:299] Negotiation successful
0812 21:07:38.814850 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260812 21:07:38.815131 11606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:38.809829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:46625 (local address 127.11.254.129:47965)
0812 21:07:38.810038 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0812 21:07:38.810081 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 21:07:38.810623 (+ 542us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:07:38.810873 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:07:38.810877 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:07:38.810886 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:07:38.811414 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:07:38.811418 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:07:38.812194 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:07:38.812195 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:07:38.813347 (+ 1152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:07:38.813350 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:07:38.813788 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:07:38.813789 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:07:38.813942 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:07:38.814531 (+ 589us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 21:07:38.814532 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:07:38.814563 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 21:07:38.814590 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:07:38.814764 (+ 174us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:07:38.814766 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:07:38.814769 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 21:07:38.814816 (+ 47us) client_negotiation.cc:240] Negotiation successful
0812 21:07:38.814903 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":88,"threads_started":1}
I20260812 21:07:38.815837 11600 heartbeater.cc:347] Connected to a master server at 127.11.254.190:46625
I20260812 21:07:38.815930 11600 heartbeater.cc:464] Registering TS with master...
I20260812 21:07:38.816114 11600 heartbeater.cc:511] Master 127.11.254.190:46625 requested a full tablet report, sending...
I20260812 21:07:38.816591 11246 ts_manager.cc:194] Registered new tserver with Master: 1cdd0b991e964392ba8e366e0d441585 (127.11.254.129:40409)
I20260812 21:07:38.817190 11246 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.254.129:47965
W20260812 21:07:38.895857 11607 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:07:38.896065 11607 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:07:38.896093 11607 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:07:38.897233 11607 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:07:38.897272 11607 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:07:38.897305 11607 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.254.130
I20260812 21:07:38.898675 11607 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.254.148:35813
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786568607354921-12282-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786568607354921-12282-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.254.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786568607354921-12282-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.254.130
--webserver_port=0
--tserver_master_addrs=127.11.254.190:46625
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11607
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.254.130
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786568607354921-12282-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:07:38.898916 11607 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:07:38.899199 11607 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:07:38.904191 11612 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:07:38.904253 11616 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:07:38.904322 11607 server_base.cc:1034] running on GCE node
W20260812 21:07:38.904577 11614 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:07:38.904814 11607 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:07:38.905069 11607 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:07:38.906214 11607 hybrid_clock.cc:648] HybridClock initialized: now 1786568858906198 us; error 30 us; skew 500 ppm
I20260812 21:07:38.907498 11607 webserver.cc:460] Webserver started at http://127.11.254.130:37667/ using document root <none> and password file <none>
I20260812 21:07:38.907733 11607 fs_manager.cc:359] Metadata directory not provided
I20260812 21:07:38.907786 11607 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:07:38.907910 11607 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:07:38.909039 11607 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786568607354921-12282-0/minicluster-data/ts-1/data/instance:
uuid: "c480ec32328045c7b3df6d6859b637b8"
format_stamp: "Formatted at 2026-08-12 21:07:38 on dist-test-slave-vpvm"
server_key: "dff08704da5dc9e397dddbbf644f26fc"
server_key_iv: "06d7762a7ca8d7dee0da2a194edcfe45"
server_key_version: "encryptionkey@0"
I20260812 21:07:38.909389 11607 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786568607354921-12282-0/minicluster-data/ts-1/wal/instance:
uuid: "c480ec32328045c7b3df6d6859b637b8"
format_stamp: "Formatted at 2026-08-12 21:07:38 on dist-test-slave-vpvm"
server_key: "dff08704da5dc9e397dddbbf644f26fc"
server_key_iv: "06d7762a7ca8d7dee0da2a194edcfe45"
server_key_version: "encryptionkey@0"
I20260812 21:07:38.910727 11607 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 21:07:38.911525 11625 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:07:38.911715 11607 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 21:07:38.911803 11607 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786568607354921-12282-0/minicluster-data/ts-1/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786568607354921-12282-0/minicluster-data/ts-1/wal
uuid: "c480ec32328045c7b3df6d6859b637b8"
format_stamp: "Formatted at 2026-08-12 21:07:38 on dist-test-slave-vpvm"
server_key: "dff08704da5dc9e397dddbbf644f26fc"
server_key_iv: "06d7762a7ca8d7dee0da2a194edcfe45"
server_key_version: "encryptionkey@0"
I20260812 21:07:38.911867 11607 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786568607354921-12282-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786568607354921-12282-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786568607354921-12282-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:07:38.965547 11607 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:07:38.965873 11607 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:07:38.966041 11607 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:07:38.966282 11607 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:07:38.966615 11607 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:07:38.966650 11607 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:07:38.966679 11607 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:07:38.966702 11607 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:07:38.972117 11607 rpc_server.cc:307] RPC server started. Bound to: 127.11.254.130:36315
I20260812 21:07:38.972184 11831 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.254.130:36315 every 8 connection(s)
I20260812 21:07:38.972491 11607 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786568607354921-12282-0/minicluster-data/ts-1/data/info.pb
I20260812 21:07:38.977630 11353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:38.973217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.130:38447 (local address 127.11.254.190:46625)
0812 21:07:38.973286 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0812 21:07:38.973288 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:07:38.973509 (+ 221us) server_negotiation.cc:407] Connection header received
0812 21:07:38.974044 (+ 535us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:07:38.974045 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:07:38.974056 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:07:38.974103 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:07:38.974768 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:38.975271 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:38.976237 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:38.976432 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:38.976495 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:07:38.977402 (+ 907us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:07:38.977403 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:07:38.977404 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 21:07:38.977408 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:07:38.977424 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 21:07:38.977428 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:07:38.977471 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0812 21:07:38.977538 (+ 67us) server_negotiation.cc:299] Negotiation successful
0812 21:07:38.977567 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260812 21:07:38.977871 11839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:38.973208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:46625 (local address 127.11.254.130:38447)
0812 21:07:38.973393 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0812 21:07:38.973427 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0812 21:07:38.973928 (+ 501us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:07:38.974168 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:07:38.974172 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:07:38.974180 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:07:38.974686 (+ 506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:07:38.974691 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:07:38.975341 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:07:38.975342 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:07:38.976143 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:07:38.976146 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:07:38.976505 (+ 359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:07:38.976506 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:07:38.976627 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:07:38.977264 (+ 637us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 21:07:38.977265 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:07:38.977295 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 21:07:38.977318 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:07:38.977481 (+ 163us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:07:38.977483 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:07:38.977486 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 21:07:38.977567 (+ 81us) client_negotiation.cc:240] Negotiation successful
0812 21:07:38.977659 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":91,"threads_started":1}
I20260812 21:07:38.978404 11832 heartbeater.cc:347] Connected to a master server at 127.11.254.190:46625
I20260812 21:07:38.978513 11832 heartbeater.cc:464] Registering TS with master...
I20260812 21:07:38.978690 11832 heartbeater.cc:511] Master 127.11.254.190:46625 requested a full tablet report, sending...
I20260812 21:07:38.979046 11246 ts_manager.cc:194] Registered new tserver with Master: c480ec32328045c7b3df6d6859b637b8 (127.11.254.130:36315)
I20260812 21:07:38.979382 11246 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.254.130:38447
I20260812 21:07:38.980353 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 11607
I20260812 21:07:38.980461 12282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786568607354921-12282-0/minicluster-data/ts-1/wal/instance
I20260812 21:07:38.980648 12282 external_mini_cluster.cc:1428] Setting key f5daad2ef077e3c9bdf7f1954e650cd6
I20260812 21:07:38.981509 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786568607354921-12282-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786568607354921-12282-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786568607354921-12282-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786568607354921-12282-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.254.131:0
--local_ip_for_outbound_sockets=127.11.254.131
--webserver_interface=127.11.254.131
--webserver_port=0
--tserver_master_addrs=127.11.254.190:46625
--builtin_ntp_servers=127.11.254.148:35813
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 21:07:39.064806 11840 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:07:39.065034 11840 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:07:39.065065 11840 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:07:39.066291 11840 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:07:39.066335 11840 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:07:39.066370 11840 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.254.131
I20260812 21:07:39.067770 11840 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.254.148:35813
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786568607354921-12282-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786568607354921-12282-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.254.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786568607354921-12282-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.254.131
--webserver_port=0
--tserver_master_addrs=127.11.254.190:46625
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11840
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.254.131
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786568607354921-12282-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:07:39.068041 11840 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:07:39.068354 11840 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:07:39.073521 11853 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:07:39.073559 11848 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:07:39.073765 11840 server_base.cc:1034] running on GCE node
W20260812 21:07:39.073593 11849 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:07:39.074026 11840 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:07:39.074262 11840 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:07:39.075424 11840 hybrid_clock.cc:648] HybridClock initialized: now 1786568859075400 us; error 38 us; skew 500 ppm
I20260812 21:07:39.076943 11840 webserver.cc:460] Webserver started at http://127.11.254.131:38969/ using document root <none> and password file <none>
I20260812 21:07:39.077186 11840 fs_manager.cc:359] Metadata directory not provided
I20260812 21:07:39.077237 11840 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:07:39.077373 11840 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:07:39.078536 11840 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786568607354921-12282-0/minicluster-data/ts-2/data/instance:
uuid: "0a74e2a39cee4a76888b5f9c4ed84cb7"
format_stamp: "Formatted at 2026-08-12 21:07:39 on dist-test-slave-vpvm"
server_key: "18b13a57d26d897bed11c3def2380082"
server_key_iv: "e91df119599cf4e8dbead722913dd447"
server_key_version: "encryptionkey@0"
I20260812 21:07:39.078887 11840 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786568607354921-12282-0/minicluster-data/ts-2/wal/instance:
uuid: "0a74e2a39cee4a76888b5f9c4ed84cb7"
format_stamp: "Formatted at 2026-08-12 21:07:39 on dist-test-slave-vpvm"
server_key: "18b13a57d26d897bed11c3def2380082"
server_key_iv: "e91df119599cf4e8dbead722913dd447"
server_key_version: "encryptionkey@0"
I20260812 21:07:39.080291 11840 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 21:07:39.081133 11862 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:07:39.081314 11840 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 21:07:39.081411 11840 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786568607354921-12282-0/minicluster-data/ts-2/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786568607354921-12282-0/minicluster-data/ts-2/wal
uuid: "0a74e2a39cee4a76888b5f9c4ed84cb7"
format_stamp: "Formatted at 2026-08-12 21:07:39 on dist-test-slave-vpvm"
server_key: "18b13a57d26d897bed11c3def2380082"
server_key_iv: "e91df119599cf4e8dbead722913dd447"
server_key_version: "encryptionkey@0"
I20260812 21:07:39.081504 11840 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786568607354921-12282-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786568607354921-12282-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786568607354921-12282-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:07:39.094264 11840 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:07:39.094578 11840 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:07:39.094692 11840 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:07:39.094902 11840 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:07:39.095208 11840 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:07:39.095240 11840 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:07:39.095266 11840 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:07:39.095283 11840 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:07:39.100788 11840 rpc_server.cc:307] RPC server started. Bound to: 127.11.254.131:36887
I20260812 21:07:39.100845 12048 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.254.131:36887 every 8 connection(s)
I20260812 21:07:39.101181 11840 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786568607354921-12282-0/minicluster-data/ts-2/data/info.pb
I20260812 21:07:39.106654 11353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:39.102010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.131:37899 (local address 127.11.254.190:46625)
0812 21:07:39.102110 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0812 21:07:39.102113 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 21:07:39.102332 (+ 219us) server_negotiation.cc:407] Connection header received
0812 21:07:39.102884 (+ 552us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:07:39.102886 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:07:39.102899 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:07:39.102951 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:07:39.103685 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:39.104203 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:39.105195 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:39.105392 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:39.105450 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:07:39.106377 (+ 927us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:07:39.106378 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:07:39.106379 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 21:07:39.106385 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:07:39.106403 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 21:07:39.106407 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:07:39.106453 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0812 21:07:39.106533 (+ 80us) server_negotiation.cc:299] Negotiation successful
0812 21:07:39.106569 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260812 21:07:39.106866 12051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:39.101975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:46625 (local address 127.11.254.131:37899)
0812 21:07:39.102199 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0812 21:07:39.102241 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 21:07:39.102777 (+ 536us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:07:39.103041 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:07:39.103044 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:07:39.103053 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:07:39.103587 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:07:39.103592 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:07:39.104291 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:07:39.104292 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:07:39.105104 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:07:39.105106 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:07:39.105464 (+ 358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:07:39.105465 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:07:39.105611 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:07:39.106217 (+ 606us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 21:07:39.106218 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:07:39.106249 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 21:07:39.106274 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:07:39.106461 (+ 187us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:07:39.106466 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:07:39.106469 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 21:07:39.106529 (+ 60us) client_negotiation.cc:240] Negotiation successful
0812 21:07:39.106647 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":91,"threads_started":1}
I20260812 21:07:39.107156 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 11840
I20260812 21:07:39.107262 12282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786568607354921-12282-0/minicluster-data/ts-2/wal/instance
I20260812 21:07:39.107374 12049 heartbeater.cc:347] Connected to a master server at 127.11.254.190:46625
I20260812 21:07:39.107470 12282 external_mini_cluster.cc:1428] Setting key 329b107df847a351c73be9f4d8122aa8
I20260812 21:07:39.107486 12049 heartbeater.cc:464] Registering TS with master...
I20260812 21:07:39.107678 12049 heartbeater.cc:511] Master 127.11.254.190:46625 requested a full tablet report, sending...
I20260812 21:07:39.108060 11246 ts_manager.cc:194] Registered new tserver with Master: 0a74e2a39cee4a76888b5f9c4ed84cb7 (127.11.254.131:36887)
I20260812 21:07:39.108409 11246 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.254.131:37899
I20260812 21:07:39.108974 12282 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 21:07:39.113160 11353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:39.110347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43014 (local address 127.11.254.190:46625)
0812 21:07:39.110388 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0812 21:07:39.110389 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 21:07:39.110515 (+ 126us) server_negotiation.cc:407] Connection header received
0812 21:07:39.110537 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:07:39.110539 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:07:39.110549 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:07:39.110594 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:07:39.110719 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:39.111180 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:39.111984 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:39.112176 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:39.112230 (+ 54us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:07:39.112897 (+ 667us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:07:39.112898 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:07:39.112898 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 21:07:39.112902 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:07:39.112916 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 21:07:39.112920 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:07:39.113009 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0812 21:07:39.113093 (+ 84us) server_negotiation.cc:299] Negotiation successful
0812 21:07:39.113104 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 21:07:39.114766 11246 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43014:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 21:07:39.116338 11246 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 21:07:39.122614 12065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:39.118674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50398 (local address 127.11.254.131:36887)
0812 21:07:39.118837 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0812 21:07:39.118842 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 21:07:39.119023 (+ 181us) server_negotiation.cc:407] Connection header received
0812 21:07:39.119073 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:07:39.119075 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:07:39.119090 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:07:39.119154 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:07:39.119410 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:39.120565 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:39.121489 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:39.122188 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:39.122265 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:07:39.122356 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0812 21:07:39.122476 (+ 120us) server_negotiation.cc:299] Negotiation successful
0812 21:07:39.122515 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":71,"threads_started":1}
I20260812 21:07:39.122605 12059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:39.118489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.131:36887 (local address 127.0.0.1:50398)
0812 21:07:39.118833 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0812 21:07:39.118878 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 21:07:39.118951 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:07:39.119231 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:07:39.119234 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:07:39.119236 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:07:39.119322 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:07:39.119325 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:07:39.120652 (+ 1327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:07:39.120653 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:07:39.121406 (+ 753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:07:39.121408 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:07:39.122271 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:07:39.122272 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:07:39.122357 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:07:39.122372 (+ 15us) client_negotiation.cc:769] Sending connection context
0812 21:07:39.122461 (+ 89us) client_negotiation.cc:240] Negotiation successful
0812 21:07:39.122516 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":320,"mutex_wait_us":4,"thread_start_us":70,"threads_started":1}
I20260812 21:07:39.122844 12067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:39.118703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.130:36315 (local address 127.0.0.1:49134)
0812 21:07:39.119088 (+ 385us) negotiation.cc:107] Waiting for socket to connect
0812 21:07:39.119097 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 21:07:39.119149 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:07:39.119534 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:07:39.119535 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:07:39.119536 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:07:39.119575 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:07:39.119576 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:07:39.120434 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:07:39.120436 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:07:39.121599 (+ 1163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:07:39.121602 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:07:39.122608 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:07:39.122610 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:07:39.122670 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:07:39.122674 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 21:07:39.122724 (+ 50us) client_negotiation.cc:240] Negotiation successful
0812 21:07:39.122763 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":370,"thread_start_us":92,"threads_started":1}
I20260812 21:07:39.123147 12069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:39.118825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49134 (local address 127.11.254.130:36315)
0812 21:07:39.119033 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0812 21:07:39.119037 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 21:07:39.119244 (+ 207us) server_negotiation.cc:407] Connection header received
0812 21:07:39.119358 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:07:39.119360 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:07:39.119389 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:07:39.119464 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:07:39.119641 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:39.120347 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:39.121689 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:39.122432 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:39.122494 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:07:39.122571 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0812 21:07:39.123003 (+ 432us) server_negotiation.cc:299] Negotiation successful
0812 21:07:39.123057 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":87,"threads_started":1}
I20260812 21:07:39.123384 11938 tablet_service.cc:1467] Processing CreateTablet for tablet d837d5009ec944a692454efa270321fc (DEFAULT_TABLE table=test-table [id=975a7a2417fd4d11bff3df51ca50e4e5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:07:39.123543 12062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:39.118497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.129:40409 (local address 127.0.0.1:37832)
0812 21:07:39.118848 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0812 21:07:39.118894 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 21:07:39.118953 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:07:39.119231 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:07:39.119234 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:07:39.119239 (+ 5us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:07:39.119322 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:07:39.119325 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:07:39.120637 (+ 1312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:07:39.120638 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:07:39.121940 (+ 1302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:07:39.121943 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:07:39.123287 (+ 1344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:07:39.123289 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:07:39.123355 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:07:39.123358 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 21:07:39.123431 (+ 73us) client_negotiation.cc:240] Negotiation successful
0812 21:07:39.123474 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":337,"mutex_wait_us":48,"thread_start_us":73,"threads_started":1}
I20260812 21:07:39.123796 11938 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d837d5009ec944a692454efa270321fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:07:39.123728 11722 tablet_service.cc:1467] Processing CreateTablet for tablet d837d5009ec944a692454efa270321fc (DEFAULT_TABLE table=test-table [id=975a7a2417fd4d11bff3df51ca50e4e5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:07:39.123953 12064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:39.118611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37832 (local address 127.11.254.129:40409)
0812 21:07:39.118917 (+ 306us) server_negotiation.cc:206] Beginning negotiation
0812 21:07:39.118921 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 21:07:39.118945 (+ 24us) server_negotiation.cc:407] Connection header received
0812 21:07:39.119043 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:07:39.119045 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:07:39.119061 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:07:39.119154 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:07:39.119403 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:39.120551 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:39.122046 (+ 1495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:39.123105 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:39.123188 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:07:39.123359 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0812 21:07:39.123739 (+ 380us) server_negotiation.cc:299] Negotiation successful
0812 21:07:39.123787 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":271,"thread_start_us":106,"threads_started":1}
I20260812 21:07:39.124069 11722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d837d5009ec944a692454efa270321fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:07:39.127249 12072 tablet_bootstrap.cc:492] T d837d5009ec944a692454efa270321fc P c480ec32328045c7b3df6d6859b637b8: Bootstrap starting.
I20260812 21:07:39.127918 12072 tablet_bootstrap.cc:654] T d837d5009ec944a692454efa270321fc P c480ec32328045c7b3df6d6859b637b8: Neither blocks nor log segments found. Creating new log.
I20260812 21:07:39.127904 11472 tablet_service.cc:1467] Processing CreateTablet for tablet d837d5009ec944a692454efa270321fc (DEFAULT_TABLE table=test-table [id=975a7a2417fd4d11bff3df51ca50e4e5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:07:39.128202 12072 log.cc:824] T d837d5009ec944a692454efa270321fc P c480ec32328045c7b3df6d6859b637b8: Log is configured to *not* fsync() on all Append() calls
I20260812 21:07:39.128222 11472 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d837d5009ec944a692454efa270321fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:07:39.128573 12071 tablet_bootstrap.cc:492] T d837d5009ec944a692454efa270321fc P 0a74e2a39cee4a76888b5f9c4ed84cb7: Bootstrap starting.
I20260812 21:07:39.128960 12072 tablet_bootstrap.cc:492] T d837d5009ec944a692454efa270321fc P c480ec32328045c7b3df6d6859b637b8: No bootstrap required, opened a new log
I20260812 21:07:39.129037 12072 ts_tablet_manager.cc:1397] T d837d5009ec944a692454efa270321fc P c480ec32328045c7b3df6d6859b637b8: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 21:07:39.129144 12071 tablet_bootstrap.cc:654] T d837d5009ec944a692454efa270321fc P 0a74e2a39cee4a76888b5f9c4ed84cb7: Neither blocks nor log segments found. Creating new log.
I20260812 21:07:39.129405 12071 log.cc:824] T d837d5009ec944a692454efa270321fc P 0a74e2a39cee4a76888b5f9c4ed84cb7: Log is configured to *not* fsync() on all Append() calls
I20260812 21:07:39.130159 12071 tablet_bootstrap.cc:492] T d837d5009ec944a692454efa270321fc P 0a74e2a39cee4a76888b5f9c4ed84cb7: No bootstrap required, opened a new log
I20260812 21:07:39.130231 12071 ts_tablet_manager.cc:1397] T d837d5009ec944a692454efa270321fc P 0a74e2a39cee4a76888b5f9c4ed84cb7: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 21:07:39.131544 12076 tablet_bootstrap.cc:492] T d837d5009ec944a692454efa270321fc P 1cdd0b991e964392ba8e366e0d441585: Bootstrap starting.
I20260812 21:07:39.131582 12072 raft_consensus.cc:348] T d837d5009ec944a692454efa270321fc P c480ec32328045c7b3df6d6859b637b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a74e2a39cee4a76888b5f9c4ed84cb7" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 36887 } } peers { permanent_uuid: "1cdd0b991e964392ba8e366e0d441585" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 40409 } } peers { permanent_uuid: "c480ec32328045c7b3df6d6859b637b8" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 36315 } }
I20260812 21:07:39.131762 12072 raft_consensus.cc:374] T d837d5009ec944a692454efa270321fc P c480ec32328045c7b3df6d6859b637b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:07:39.131812 12072 raft_consensus.cc:729] T d837d5009ec944a692454efa270321fc P c480ec32328045c7b3df6d6859b637b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c480ec32328045c7b3df6d6859b637b8, State: Initialized, Role: FOLLOWER
I20260812 21:07:39.131929 12072 consensus_queue.cc:260] T d837d5009ec944a692454efa270321fc P c480ec32328045c7b3df6d6859b637b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a74e2a39cee4a76888b5f9c4ed84cb7" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 36887 } } peers { permanent_uuid: "1cdd0b991e964392ba8e366e0d441585" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 40409 } } peers { permanent_uuid: "c480ec32328045c7b3df6d6859b637b8" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 36315 } }
I20260812 21:07:39.132185 12072 ts_tablet_manager.cc:1428] T d837d5009ec944a692454efa270321fc P c480ec32328045c7b3df6d6859b637b8: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 21:07:39.132258 11832 heartbeater.cc:503] Master 127.11.254.190:46625 was elected leader, sending a full tablet report...
I20260812 21:07:39.132354 12076 tablet_bootstrap.cc:654] T d837d5009ec944a692454efa270321fc P 1cdd0b991e964392ba8e366e0d441585: Neither blocks nor log segments found. Creating new log.
I20260812 21:07:39.132668 12076 log.cc:824] T d837d5009ec944a692454efa270321fc P 1cdd0b991e964392ba8e366e0d441585: Log is configured to *not* fsync() on all Append() calls
I20260812 21:07:39.132889 12071 raft_consensus.cc:348] T d837d5009ec944a692454efa270321fc P 0a74e2a39cee4a76888b5f9c4ed84cb7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a74e2a39cee4a76888b5f9c4ed84cb7" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 36887 } } peers { permanent_uuid: "1cdd0b991e964392ba8e366e0d441585" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 40409 } } peers { permanent_uuid: "c480ec32328045c7b3df6d6859b637b8" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 36315 } }
I20260812 21:07:39.133026 12071 raft_consensus.cc:374] T d837d5009ec944a692454efa270321fc P 0a74e2a39cee4a76888b5f9c4ed84cb7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:07:39.133059 12071 raft_consensus.cc:729] T d837d5009ec944a692454efa270321fc P 0a74e2a39cee4a76888b5f9c4ed84cb7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a74e2a39cee4a76888b5f9c4ed84cb7, State: Initialized, Role: FOLLOWER
I20260812 21:07:39.133169 12071 consensus_queue.cc:260] T d837d5009ec944a692454efa270321fc P 0a74e2a39cee4a76888b5f9c4ed84cb7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a74e2a39cee4a76888b5f9c4ed84cb7" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 36887 } } peers { permanent_uuid: "1cdd0b991e964392ba8e366e0d441585" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 40409 } } peers { permanent_uuid: "c480ec32328045c7b3df6d6859b637b8" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 36315 } }
I20260812 21:07:39.133412 12071 ts_tablet_manager.cc:1428] T d837d5009ec944a692454efa270321fc P 0a74e2a39cee4a76888b5f9c4ed84cb7: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 21:07:39.133422 12076 tablet_bootstrap.cc:492] T d837d5009ec944a692454efa270321fc P 1cdd0b991e964392ba8e366e0d441585: No bootstrap required, opened a new log
I20260812 21:07:39.133489 12076 ts_tablet_manager.cc:1397] T d837d5009ec944a692454efa270321fc P 1cdd0b991e964392ba8e366e0d441585: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 21:07:39.133553 12049 heartbeater.cc:503] Master 127.11.254.190:46625 was elected leader, sending a full tablet report...
I20260812 21:07:39.135535 12076 raft_consensus.cc:348] T d837d5009ec944a692454efa270321fc P 1cdd0b991e964392ba8e366e0d441585 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a74e2a39cee4a76888b5f9c4ed84cb7" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 36887 } } peers { permanent_uuid: "1cdd0b991e964392ba8e366e0d441585" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 40409 } } peers { permanent_uuid: "c480ec32328045c7b3df6d6859b637b8" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 36315 } }
I20260812 21:07:39.135671 12076 raft_consensus.cc:374] T d837d5009ec944a692454efa270321fc P 1cdd0b991e964392ba8e366e0d441585 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:07:39.135691 12076 raft_consensus.cc:729] T d837d5009ec944a692454efa270321fc P 1cdd0b991e964392ba8e366e0d441585 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1cdd0b991e964392ba8e366e0d441585, State: Initialized, Role: FOLLOWER
I20260812 21:07:39.135772 12076 consensus_queue.cc:260] T d837d5009ec944a692454efa270321fc P 1cdd0b991e964392ba8e366e0d441585 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a74e2a39cee4a76888b5f9c4ed84cb7" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 36887 } } peers { permanent_uuid: "1cdd0b991e964392ba8e366e0d441585" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 40409 } } peers { permanent_uuid: "c480ec32328045c7b3df6d6859b637b8" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 36315 } }
I20260812 21:07:39.135987 12076 ts_tablet_manager.cc:1428] T d837d5009ec944a692454efa270321fc P 1cdd0b991e964392ba8e366e0d441585: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260812 21:07:39.136072 11600 heartbeater.cc:503] Master 127.11.254.190:46625 was elected leader, sending a full tablet report...
W20260812 21:07:39.223093 11837 tablet.cc:2369] T d837d5009ec944a692454efa270321fc P c480ec32328045c7b3df6d6859b637b8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 21:07:39.228154 12080 raft_consensus.cc:482] T d837d5009ec944a692454efa270321fc P 0a74e2a39cee4a76888b5f9c4ed84cb7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 21:07:39.228293 12080 raft_consensus.cc:504] T d837d5009ec944a692454efa270321fc P 0a74e2a39cee4a76888b5f9c4ed84cb7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a74e2a39cee4a76888b5f9c4ed84cb7" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 36887 } } peers { permanent_uuid: "1cdd0b991e964392ba8e366e0d441585" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 40409 } } peers { permanent_uuid: "c480ec32328045c7b3df6d6859b637b8" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 36315 } }
I20260812 21:07:39.228659 12080 leader_election.cc:288] T d837d5009ec944a692454efa270321fc P 0a74e2a39cee4a76888b5f9c4ed84cb7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1cdd0b991e964392ba8e366e0d441585 (127.11.254.129:40409), c480ec32328045c7b3df6d6859b637b8 (127.11.254.130:36315)
I20260812 21:07:39.231688 12064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:39.228972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.131:56741 (local address 127.11.254.129:40409)
0812 21:07:39.229033 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0812 21:07:39.229035 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:07:39.229049 (+ 14us) server_negotiation.cc:407] Connection header received
0812 21:07:39.229114 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:07:39.229115 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:07:39.229127 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:07:39.229203 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:07:39.229406 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:39.229925 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:39.230976 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:39.231348 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:39.231424 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:07:39.231434 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0812 21:07:39.231571 (+ 137us) server_negotiation.cc:299] Negotiation successful
0812 21:07:39.231614 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260812 21:07:39.231729 12069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:39.229124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.131:33693 (local address 127.11.254.130:36315)
0812 21:07:39.229181 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0812 21:07:39.229183 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:07:39.229312 (+ 129us) server_negotiation.cc:407] Connection header received
0812 21:07:39.229346 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:07:39.229347 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:07:39.229360 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:07:39.229408 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:07:39.229558 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:39.230070 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:39.231002 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:39.231381 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:39.231443 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:07:39.231451 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 21:07:39.231630 (+ 179us) server_negotiation.cc:299] Negotiation successful
0812 21:07:39.231670 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 21:07:39.231721 12051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:39.228911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.129:40409 (local address 127.11.254.131:56741)
0812 21:07:39.228971 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0812 21:07:39.228985 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 21:07:39.229039 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:07:39.229234 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:07:39.229235 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:07:39.229237 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:07:39.229308 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:07:39.229310 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:07:39.230042 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:07:39.230043 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:07:39.230867 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:07:39.230870 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:07:39.231432 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:07:39.231433 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:07:39.231500 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:07:39.231504 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 21:07:39.231557 (+ 53us) client_negotiation.cc:240] Negotiation successful
0812 21:07:39.231605 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260812 21:07:39.231737 12083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:39.229045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.130:36315 (local address 127.11.254.131:33693)
0812 21:07:39.229235 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0812 21:07:39.229246 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 21:07:39.229303 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:07:39.229418 (+ 115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:07:39.229418 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:07:39.229420 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:07:39.229494 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:07:39.229496 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:07:39.230139 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:07:39.230140 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:07:39.230908 (+ 768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:07:39.230910 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:07:39.231454 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:07:39.231455 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:07:39.231537 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:07:39.231540 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 21:07:39.231585 (+ 45us) client_negotiation.cc:240] Negotiation successful
0812 21:07:39.231625 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"mutex_wait_us":12,"thread_start_us":127,"threads_started":1}
I20260812 21:07:39.232161 11506 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d837d5009ec944a692454efa270321fc" candidate_uuid: "0a74e2a39cee4a76888b5f9c4ed84cb7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cdd0b991e964392ba8e366e0d441585" is_pre_election: true
I20260812 21:07:39.232324 11506 raft_consensus.cc:2454] T d837d5009ec944a692454efa270321fc P 1cdd0b991e964392ba8e366e0d441585 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a74e2a39cee4a76888b5f9c4ed84cb7 in term 0.
I20260812 21:07:39.232367 11758 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d837d5009ec944a692454efa270321fc" candidate_uuid: "0a74e2a39cee4a76888b5f9c4ed84cb7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c480ec32328045c7b3df6d6859b637b8" is_pre_election: true
I20260812 21:07:39.232574 11758 raft_consensus.cc:2454] T d837d5009ec944a692454efa270321fc P c480ec32328045c7b3df6d6859b637b8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a74e2a39cee4a76888b5f9c4ed84cb7 in term 0.
I20260812 21:07:39.232578 11864 leader_election.cc:302] T d837d5009ec944a692454efa270321fc P 0a74e2a39cee4a76888b5f9c4ed84cb7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a74e2a39cee4a76888b5f9c4ed84cb7, 1cdd0b991e964392ba8e366e0d441585; no voters:
I20260812 21:07:39.232710 12080 raft_consensus.cc:2781] T d837d5009ec944a692454efa270321fc P 0a74e2a39cee4a76888b5f9c4ed84cb7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 21:07:39.232766 12080 raft_consensus.cc:482] T d837d5009ec944a692454efa270321fc P 0a74e2a39cee4a76888b5f9c4ed84cb7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 21:07:39.232781 12080 raft_consensus.cc:3037] T d837d5009ec944a692454efa270321fc P 0a74e2a39cee4a76888b5f9c4ed84cb7 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:07:39.233520 12080 raft_consensus.cc:504] T d837d5009ec944a692454efa270321fc P 0a74e2a39cee4a76888b5f9c4ed84cb7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a74e2a39cee4a76888b5f9c4ed84cb7" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 36887 } } peers { permanent_uuid: "1cdd0b991e964392ba8e366e0d441585" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 40409 } } peers { permanent_uuid: "c480ec32328045c7b3df6d6859b637b8" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 36315 } }
I20260812 21:07:39.233630 12080 leader_election.cc:288] T d837d5009ec944a692454efa270321fc P 0a74e2a39cee4a76888b5f9c4ed84cb7 [CANDIDATE]: Term 1 election: Requested vote from peers 1cdd0b991e964392ba8e366e0d441585 (127.11.254.129:40409), c480ec32328045c7b3df6d6859b637b8 (127.11.254.130:36315)
I20260812 21:07:39.233820 11506 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d837d5009ec944a692454efa270321fc" candidate_uuid: "0a74e2a39cee4a76888b5f9c4ed84cb7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cdd0b991e964392ba8e366e0d441585"
I20260812 21:07:39.233876 11758 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d837d5009ec944a692454efa270321fc" candidate_uuid: "0a74e2a39cee4a76888b5f9c4ed84cb7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c480ec32328045c7b3df6d6859b637b8"
I20260812 21:07:39.233916 11506 raft_consensus.cc:3037] T d837d5009ec944a692454efa270321fc P 1cdd0b991e964392ba8e366e0d441585 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:07:39.233975 11758 raft_consensus.cc:3037] T d837d5009ec944a692454efa270321fc P c480ec32328045c7b3df6d6859b637b8 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:07:39.234771 11758 raft_consensus.cc:2454] T d837d5009ec944a692454efa270321fc P c480ec32328045c7b3df6d6859b637b8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a74e2a39cee4a76888b5f9c4ed84cb7 in term 1.
I20260812 21:07:39.234786 11506 raft_consensus.cc:2454] T d837d5009ec944a692454efa270321fc P 1cdd0b991e964392ba8e366e0d441585 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a74e2a39cee4a76888b5f9c4ed84cb7 in term 1.
I20260812 21:07:39.234938 11864 leader_election.cc:302] T d837d5009ec944a692454efa270321fc P 0a74e2a39cee4a76888b5f9c4ed84cb7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a74e2a39cee4a76888b5f9c4ed84cb7, c480ec32328045c7b3df6d6859b637b8; no voters:
I20260812 21:07:39.235023 12080 raft_consensus.cc:2781] T d837d5009ec944a692454efa270321fc P 0a74e2a39cee4a76888b5f9c4ed84cb7 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:07:39.235169 12080 raft_consensus.cc:686] T d837d5009ec944a692454efa270321fc P 0a74e2a39cee4a76888b5f9c4ed84cb7 [term 1 LEADER]: Becoming Leader. State: Replica: 0a74e2a39cee4a76888b5f9c4ed84cb7, State: Running, Role: LEADER
I20260812 21:07:39.235251 12080 consensus_queue.cc:237] T d837d5009ec944a692454efa270321fc P 0a74e2a39cee4a76888b5f9c4ed84cb7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a74e2a39cee4a76888b5f9c4ed84cb7" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 36887 } } peers { permanent_uuid: "1cdd0b991e964392ba8e366e0d441585" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 40409 } } peers { permanent_uuid: "c480ec32328045c7b3df6d6859b637b8" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 36315 } }
I20260812 21:07:39.236100 11245 catalog_manager.cc:5495] T d837d5009ec944a692454efa270321fc P 0a74e2a39cee4a76888b5f9c4ed84cb7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0a74e2a39cee4a76888b5f9c4ed84cb7 (127.11.254.131). New cstate: current_term: 1 leader_uuid: "0a74e2a39cee4a76888b5f9c4ed84cb7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a74e2a39cee4a76888b5f9c4ed84cb7" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 36887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1cdd0b991e964392ba8e366e0d441585" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 40409 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c480ec32328045c7b3df6d6859b637b8" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 36315 } health_report { overall_health: UNKNOWN } } }
I20260812 21:07:39.267793 12065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:39.265475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50406 (local address 127.11.254.131:36887)
0812 21:07:39.265527 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0812 21:07:39.265529 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:07:39.265536 (+ 7us) server_negotiation.cc:407] Connection header received
0812 21:07:39.265572 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:07:39.265572 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:07:39.265595 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:07:39.265656 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 21:07:39.265814 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:39.266308 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:39.267072 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:39.267270 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:39.267322 (+ 52us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:07:39.267469 (+ 147us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 21:07:39.267598 (+ 129us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 21:07:39.267677 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0812 21:07:39.267701 (+ 24us) server_negotiation.cc:299] Negotiation successful
0812 21:07:39.267709 (+ 8us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 21:07:39.269702 11758 raft_consensus.cc:1261] T d837d5009ec944a692454efa270321fc P c480ec32328045c7b3df6d6859b637b8 [term 1 FOLLOWER]: Refusing update from remote peer 0a74e2a39cee4a76888b5f9c4ed84cb7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 21:07:39.269697 11506 raft_consensus.cc:1261] T d837d5009ec944a692454efa270321fc P 1cdd0b991e964392ba8e366e0d441585 [term 1 FOLLOWER]: Refusing update from remote peer 0a74e2a39cee4a76888b5f9c4ed84cb7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 21:07:39.270087 12080 consensus_queue.cc:1035] T d837d5009ec944a692454efa270321fc P 0a74e2a39cee4a76888b5f9c4ed84cb7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c480ec32328045c7b3df6d6859b637b8" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 36315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 21:07:39.270233 12084 consensus_queue.cc:1035] T d837d5009ec944a692454efa270321fc P 0a74e2a39cee4a76888b5f9c4ed84cb7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1cdd0b991e964392ba8e366e0d441585" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 40409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 21:07:39.273205 12098 mvcc.cc:205] Tried to move back new op lower bound from 7317786047565811712 to 7317786047428280320. Current Snapshot: MvccSnapshot[applied={T|T < 7317786047565811712}]
I20260812 21:07:39.273610 12095 mvcc.cc:205] Tried to move back new op lower bound from 7317786047565811712 to 7317786047428280320. Current Snapshot: MvccSnapshot[applied={T|T < 7317786047565811712 or (T in {7317786047565811712})}]
I20260812 21:07:39.275010 11246 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43014:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 21:07:39.275112 11246 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43014:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 21:07:39.275931 11246 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bec4359db5a54b9fbd8ce26632e9432f: Sending DeleteTablet for 3 replicas of tablet d837d5009ec944a692454efa270321fc
I20260812 21:07:39.276311 11722 tablet_service.cc:1514] Processing DeleteTablet for tablet d837d5009ec944a692454efa270321fc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:07:39 UTC) from {username='slave'} at 127.0.0.1:49134
I20260812 21:07:39.276320 11472 tablet_service.cc:1514] Processing DeleteTablet for tablet d837d5009ec944a692454efa270321fc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:07:39 UTC) from {username='slave'} at 127.0.0.1:37832
I20260812 21:07:39.276579 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 11362
I20260812 21:07:39.276508 11938 tablet_service.cc:1514] Processing DeleteTablet for tablet d837d5009ec944a692454efa270321fc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:07:39 UTC) from {username='slave'} at 127.0.0.1:50398
I20260812 21:07:39.283950 12120 tablet_replica.cc:330] T d837d5009ec944a692454efa270321fc P 0a74e2a39cee4a76888b5f9c4ed84cb7: stopping tablet replica
I20260812 21:07:39.284118 12120 raft_consensus.cc:2229] T d837d5009ec944a692454efa270321fc P 0a74e2a39cee4a76888b5f9c4ed84cb7 [term 1 LEADER]: Raft consensus shutting down.
I20260812 21:07:39.284267 12120 raft_consensus.cc:2258] T d837d5009ec944a692454efa270321fc P 0a74e2a39cee4a76888b5f9c4ed84cb7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 21:07:39.287612 12120 ts_tablet_manager.cc:1905] T d837d5009ec944a692454efa270321fc P 0a74e2a39cee4a76888b5f9c4ed84cb7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 21:07:39.294091 12117 tablet_replica.cc:330] T d837d5009ec944a692454efa270321fc P c480ec32328045c7b3df6d6859b637b8: stopping tablet replica
I20260812 21:07:39.294132 12120 ts_tablet_manager.cc:1918] T d837d5009ec944a692454efa270321fc P 0a74e2a39cee4a76888b5f9c4ed84cb7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 21:07:39.294209 12120 log.cc:1196] T d837d5009ec944a692454efa270321fc P 0a74e2a39cee4a76888b5f9c4ed84cb7: Deleting WAL directory at /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786568607354921-12282-0/minicluster-data/ts-2/wal/wals/d837d5009ec944a692454efa270321fc
I20260812 21:07:39.294255 12117 raft_consensus.cc:2229] T d837d5009ec944a692454efa270321fc P c480ec32328045c7b3df6d6859b637b8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 21:07:39.294387 12117 raft_consensus.cc:2258] T d837d5009ec944a692454efa270321fc P c480ec32328045c7b3df6d6859b637b8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 21:07:39.294473 12120 ts_tablet_manager.cc:1939] T d837d5009ec944a692454efa270321fc P 0a74e2a39cee4a76888b5f9c4ed84cb7: Deleting consensus metadata
I20260812 21:07:39.294960 11221 catalog_manager.cc:4841] TS 0a74e2a39cee4a76888b5f9c4ed84cb7 (127.11.254.131:36887): tablet d837d5009ec944a692454efa270321fc (table test-table [id=975a7a2417fd4d11bff3df51ca50e4e5]) successfully deleted
I20260812 21:07:39.295037 12117 ts_tablet_manager.cc:1905] T d837d5009ec944a692454efa270321fc P c480ec32328045c7b3df6d6859b637b8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 21:07:39.296658 12117 ts_tablet_manager.cc:1918] T d837d5009ec944a692454efa270321fc P c480ec32328045c7b3df6d6859b637b8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 21:07:39.296742 12117 log.cc:1196] T d837d5009ec944a692454efa270321fc P c480ec32328045c7b3df6d6859b637b8: Deleting WAL directory at /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786568607354921-12282-0/minicluster-data/ts-1/wal/wals/d837d5009ec944a692454efa270321fc
I20260812 21:07:39.297009 12117 ts_tablet_manager.cc:1939] T d837d5009ec944a692454efa270321fc P c480ec32328045c7b3df6d6859b637b8: Deleting consensus metadata
I20260812 21:07:39.297474 11222 catalog_manager.cc:4841] TS c480ec32328045c7b3df6d6859b637b8 (127.11.254.130:36315): tablet d837d5009ec944a692454efa270321fc (table test-table [id=975a7a2417fd4d11bff3df51ca50e4e5]) successfully deleted
W20260812 21:07:39.301113 11222 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.254.129:40409 (error 108)
I20260812 21:07:39.301415 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 11607
I20260812 21:07:39.304082 12067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:39.301541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.129:40409 (local address 127.0.0.1:37838)
0812 21:07:39.303990 (+ 2449us) negotiation.cc:107] Waiting for socket to connect
0812 21:07:39.304049 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.129:40409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2418}
W20260812 21:07:39.304211 11222 catalog_manager.cc:4584] TS 1cdd0b991e964392ba8e366e0d441585 (127.11.254.129:40409): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d837d5009ec944a692454efa270321fc: Network error: Client connection negotiation failed: client connection to 127.11.254.129:40409: connect: Connection refused (error 111)
I20260812 21:07:39.315528 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 11840
I20260812 21:07:39.331619 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 11197
2026-08-12T21:07:39Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (850 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568607354921-12282-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 21:07:39 dist-test-slave-vpvm krb5kdc[12124](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 21:07:39 dist-test-slave-vpvm krb5kdc[12124](info): set up 2 sockets
Aug 12 21:07:39 dist-test-slave-vpvm krb5kdc[12124](info): commencing operation
krb5kdc: starting...
W20260812 21:07:40.521314 12282 mini_kdc.cc:121] Time spent starting KDC: real 1.160s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568607354921-12282-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568607354921-12282-0/krb5kdc/test-user.keytab.
Aug 12 21:07:40 dist-test-slave-vpvm krb5kdc[12124](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568860, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T21:07:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T21:07:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.254.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.254.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.254.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.190.keytab.
Entry for principal kudu/127.11.254.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.190.keytab.
I20260812 21:07:40.649485 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568607354921-12282-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568607354921-12282-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568607354921-12282-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.254.190:35453
--webserver_interface=127.11.254.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.190.keytab
--principal=kudu/127.11.254.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.254.148:44565
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.254.190:35453
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568607354921-12282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568607354921-12282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568607354921-12282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:07:40.732450 12147 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:07:40.732681 12147 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:07:40.732712 12147 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:07:40.733870 12147 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 21:07:40.733915 12147 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:07:40.733932 12147 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:07:40.733981 12147 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 21:07:40.733999 12147 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 21:07:40.735548 12147 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.254.148:44565
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568607354921-12282-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568607354921-12282-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.254.190:35453
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.190.keytab
--principal=kudu/127.11.254.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.254.190:35453
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.254.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12147
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568607354921-12282-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:07:40.735819 12147 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:07:40.736052 12147 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:07:40.741146 12160 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:07:40.741223 12165 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:07:40.741168 12163 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:07:40.741433 12147 server_base.cc:1034] running on GCE node
I20260812 21:07:40.741720 12147 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:07:40.742093 12147 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:07:40.743271 12147 hybrid_clock.cc:648] HybridClock initialized: now 1786568860743250 us; error 38 us; skew 500 ppm
Aug 12 21:07:40 dist-test-slave-vpvm krb5kdc[12124](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568860, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:07:40.746184 12147 init.cc:377] Logged in from keytab as kudu/127.11.254.190@KRBTEST.COM (short username kudu)
I20260812 21:07:40.746677 12147 webserver.cc:460] Webserver started at http://127.11.254.190:45103/ using document root <none> and password file <none>
I20260812 21:07:40.746899 12147 fs_manager.cc:359] Metadata directory not provided
I20260812 21:07:40.746943 12147 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:07:40.747076 12147 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:07:40.748194 12147 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568607354921-12282-0/minicluster-data/master-0/data/instance:
uuid: "168985b0467b438db3e7ca3e87f4662c"
format_stamp: "Formatted at 2026-08-12 21:07:40 on dist-test-slave-vpvm"
server_key: "f8852119da3ae1dcccbbd181dcc4088a"
server_key_iv: "46032378829005a8f60ef7d27b8053e9"
server_key_version: "encryptionkey@0"
I20260812 21:07:40.748566 12147 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568607354921-12282-0/minicluster-data/master-0/wal/instance:
uuid: "168985b0467b438db3e7ca3e87f4662c"
format_stamp: "Formatted at 2026-08-12 21:07:40 on dist-test-slave-vpvm"
server_key: "f8852119da3ae1dcccbbd181dcc4088a"
server_key_iv: "46032378829005a8f60ef7d27b8053e9"
server_key_version: "encryptionkey@0"
I20260812 21:07:40.749979 12147 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260812 21:07:40.750766 12176 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:07:40.750934 12147 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 21:07:40.751027 12147 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568607354921-12282-0/minicluster-data/master-0/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568607354921-12282-0/minicluster-data/master-0/wal
uuid: "168985b0467b438db3e7ca3e87f4662c"
format_stamp: "Formatted at 2026-08-12 21:07:40 on dist-test-slave-vpvm"
server_key: "f8852119da3ae1dcccbbd181dcc4088a"
server_key_iv: "46032378829005a8f60ef7d27b8053e9"
server_key_version: "encryptionkey@0"
I20260812 21:07:40.751091 12147 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568607354921-12282-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568607354921-12282-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568607354921-12282-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:07:40.768075 12147 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:07:40.770404 12147 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:07:40.770558 12147 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:07:40.774564 12147 rpc_server.cc:307] RPC server started. Bound to: 127.11.254.190:35453
I20260812 21:07:40.774693 12262 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.254.190:35453 every 8 connection(s)
I20260812 21:07:40.774977 12147 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
I20260812 21:07:40.775301 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 12147
I20260812 21:07:40.775513 12282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568607354921-12282-0/minicluster-data/master-0/wal/instance
I20260812 21:07:40.775745 12282 external_mini_cluster.cc:1428] Setting key d2af0b33f010cbf6e691fbabf6ee22a0
I20260812 21:07:40.776113 12265 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:07:40.779448 12265 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 168985b0467b438db3e7ca3e87f4662c: Bootstrap starting.
I20260812 21:07:40.780212 12265 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 168985b0467b438db3e7ca3e87f4662c: Neither blocks nor log segments found. Creating new log.
I20260812 21:07:40.780512 12265 log.cc:824] T 00000000000000000000000000000000 P 168985b0467b438db3e7ca3e87f4662c: Log is configured to *not* fsync() on all Append() calls
I20260812 21:07:40.781195 12265 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 168985b0467b438db3e7ca3e87f4662c: No bootstrap required, opened a new log
Aug 12 21:07:40 dist-test-slave-vpvm krb5kdc[12124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568860, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.254.190@KRBTEST.COM
I20260812 21:07:40.783296 12265 raft_consensus.cc:348] T 00000000000000000000000000000000 P 168985b0467b438db3e7ca3e87f4662c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "168985b0467b438db3e7ca3e87f4662c" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 35453 } }
I20260812 21:07:40.783442 12265 raft_consensus.cc:374] T 00000000000000000000000000000000 P 168985b0467b438db3e7ca3e87f4662c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:07:40.783471 12265 raft_consensus.cc:729] T 00000000000000000000000000000000 P 168985b0467b438db3e7ca3e87f4662c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 168985b0467b438db3e7ca3e87f4662c, State: Initialized, Role: FOLLOWER
I20260812 21:07:40.783600 12265 consensus_queue.cc:260] T 00000000000000000000000000000000 P 168985b0467b438db3e7ca3e87f4662c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "168985b0467b438db3e7ca3e87f4662c" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 35453 } }
I20260812 21:07:40.783682 12265 raft_consensus.cc:388] T 00000000000000000000000000000000 P 168985b0467b438db3e7ca3e87f4662c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 21:07:40.783720 12265 raft_consensus.cc:482] T 00000000000000000000000000000000 P 168985b0467b438db3e7ca3e87f4662c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 21:07:40.783771 12265 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 168985b0467b438db3e7ca3e87f4662c [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:07:40.784402 12265 raft_consensus.cc:504] T 00000000000000000000000000000000 P 168985b0467b438db3e7ca3e87f4662c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "168985b0467b438db3e7ca3e87f4662c" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 35453 } }
I20260812 21:07:40.784497 12265 leader_election.cc:302] T 00000000000000000000000000000000 P 168985b0467b438db3e7ca3e87f4662c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 168985b0467b438db3e7ca3e87f4662c; no voters:
I20260812 21:07:40.784682 12265 leader_election.cc:288] T 00000000000000000000000000000000 P 168985b0467b438db3e7ca3e87f4662c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 21:07:40.784725 12273 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 168985b0467b438db3e7ca3e87f4662c [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:07:40.784838 12273 raft_consensus.cc:686] T 00000000000000000000000000000000 P 168985b0467b438db3e7ca3e87f4662c [term 1 LEADER]: Becoming Leader. State: Replica: 168985b0467b438db3e7ca3e87f4662c, State: Running, Role: LEADER
I20260812 21:07:40.784933 12273 consensus_queue.cc:237] T 00000000000000000000000000000000 P 168985b0467b438db3e7ca3e87f4662c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "168985b0467b438db3e7ca3e87f4662c" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 35453 } }
I20260812 21:07:40.784969 12265 sys_catalog.cc:564] T 00000000000000000000000000000000 P 168985b0467b438db3e7ca3e87f4662c [sys.catalog]: configured and running, proceeding with master startup.
I20260812 21:07:40.785290 12274 sys_catalog.cc:455] T 00000000000000000000000000000000 P 168985b0467b438db3e7ca3e87f4662c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 168985b0467b438db3e7ca3e87f4662c. Latest consensus state: current_term: 1 leader_uuid: "168985b0467b438db3e7ca3e87f4662c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "168985b0467b438db3e7ca3e87f4662c" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 35453 } } }
I20260812 21:07:40.785356 12274 sys_catalog.cc:458] T 00000000000000000000000000000000 P 168985b0467b438db3e7ca3e87f4662c [sys.catalog]: This master's current role is: LEADER
I20260812 21:07:40.785559 12280 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 21:07:40.786286 12280 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 21:07:40.787206 12268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:40.776308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55724 (local address 127.11.254.190:35453)
0812 21:07:40.776483 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0812 21:07:40.776492 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 21:07:40.776518 (+ 26us) server_negotiation.cc:407] Connection header received
0812 21:07:40.777196 (+ 678us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:07:40.777205 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:07:40.777258 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:07:40.777378 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:07:40.778256 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:40.779215 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:40.780259 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:40.780733 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:40.780836 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:07:40.783244 (+ 2408us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:07:40.783248 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:07:40.783248 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:07:40.783258 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:07:40.785095 (+ 1837us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:07:40.785489 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:07:40.785492 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:07:40.785493 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:07:40.785621 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:07:40.785858 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:07:40.785858 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:07:40.785859 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:07:40.786195 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:07:40.786761 (+ 566us) server_negotiation.cc:1035] Waiting for connection context
0812 21:07:40.786836 (+ 75us) server_negotiation.cc:299] Negotiation successful
0812 21:07:40.786980 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":73,"threads_started":1}
I20260812 21:07:40.788034 12277 sys_catalog.cc:455] T 00000000000000000000000000000000 P 168985b0467b438db3e7ca3e87f4662c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "168985b0467b438db3e7ca3e87f4662c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "168985b0467b438db3e7ca3e87f4662c" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 35453 } } }
I20260812 21:07:40.788118 12277 sys_catalog.cc:458] T 00000000000000000000000000000000 P 168985b0467b438db3e7ca3e87f4662c [sys.catalog]: This master's current role is: LEADER
I20260812 21:07:40.790452 12280 catalog_manager.cc:1347] Generated new cluster ID: b4d2a6e6fd1e4844938e82eb1066bf1b
I20260812 21:07:40.790503 12280 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 21:07:40.796447 12280 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 21:07:40.797192 12280 catalog_manager.cc:1504] Loading token signing keys...
I20260812 21:07:40.800122 12280 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 168985b0467b438db3e7ca3e87f4662c: Generated new TSK 0
I20260812 21:07:40.800287 12280 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.254.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.254.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.254.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.129.keytab.
Entry for principal kudu/127.11.254.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.129.keytab.
I20260812 21:07:40.833127 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568607354921-12282-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568607354921-12282-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568607354921-12282-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568607354921-12282-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.254.129:0
--local_ip_for_outbound_sockets=127.11.254.129
--webserver_interface=127.11.254.129
--webserver_port=0
--tserver_master_addrs=127.11.254.190:35453
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.129.keytab
--principal=kudu/127.11.254.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.254.148:44565
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568607354921-12282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568607354921-12282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568607354921-12282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:07:40.917696 12320 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:07:40.917906 12320 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:07:40.917929 12320 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:07:40.919134 12320 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:07:40.919178 12320 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:07:40.919204 12320 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.254.129
I20260812 21:07:40.920619 12320 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.254.148:44565
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568607354921-12282-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568607354921-12282-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.129.keytab
--principal=kudu/127.11.254.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.254.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568607354921-12282-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.254.129
--webserver_port=0
--tserver_master_addrs=127.11.254.190:35453
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12320
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.254.129
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568607354921-12282-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:07:40.920819 12320 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:07:40.921008 12320 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:07:40.926286 12332 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:07:40.926293 12334 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:07:40.926355 12320 server_base.cc:1034] running on GCE node
W20260812 21:07:40.926291 12337 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:07:40.926688 12320 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:07:40.926905 12320 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:07:40.928062 12320 hybrid_clock.cc:648] HybridClock initialized: now 1786568860928047 us; error 31 us; skew 500 ppm
Aug 12 21:07:40 dist-test-slave-vpvm krb5kdc[12124](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568860, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:07:40.930610 12320 init.cc:377] Logged in from keytab as kudu/127.11.254.129@KRBTEST.COM (short username kudu)
I20260812 21:07:40.931079 12320 webserver.cc:460] Webserver started at http://127.11.254.129:39971/ using document root <none> and password file <none>
I20260812 21:07:40.931310 12320 fs_manager.cc:359] Metadata directory not provided
I20260812 21:07:40.931356 12320 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:07:40.931514 12320 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:07:40.932593 12320 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568607354921-12282-0/minicluster-data/ts-0/data/instance:
uuid: "01ae9577f1e04432b4d5c3391f383e6e"
format_stamp: "Formatted at 2026-08-12 21:07:40 on dist-test-slave-vpvm"
server_key: "3884aeb6813293ffeaf36af5ab9ec355"
server_key_iv: "776fe740982163f9a651578bb357da7a"
server_key_version: "encryptionkey@0"
I20260812 21:07:40.932940 12320 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568607354921-12282-0/minicluster-data/ts-0/wal/instance:
uuid: "01ae9577f1e04432b4d5c3391f383e6e"
format_stamp: "Formatted at 2026-08-12 21:07:40 on dist-test-slave-vpvm"
server_key: "3884aeb6813293ffeaf36af5ab9ec355"
server_key_iv: "776fe740982163f9a651578bb357da7a"
server_key_version: "encryptionkey@0"
I20260812 21:07:40.934281 12320 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260812 21:07:40.935052 12350 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:07:40.935184 12320 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 21:07:40.935268 12320 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568607354921-12282-0/minicluster-data/ts-0/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568607354921-12282-0/minicluster-data/ts-0/wal
uuid: "01ae9577f1e04432b4d5c3391f383e6e"
format_stamp: "Formatted at 2026-08-12 21:07:40 on dist-test-slave-vpvm"
server_key: "3884aeb6813293ffeaf36af5ab9ec355"
server_key_iv: "776fe740982163f9a651578bb357da7a"
server_key_version: "encryptionkey@0"
I20260812 21:07:40.935335 12320 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568607354921-12282-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568607354921-12282-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568607354921-12282-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:07:40.962554 12320 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:07:40.964716 12320 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:07:40.964847 12320 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:07:40.965040 12320 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:07:40.965307 12320 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:07:40.965338 12320 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:07:40.965359 12320 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:07:40.965370 12320 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:07:40.971251 12320 rpc_server.cc:307] RPC server started. Bound to: 127.11.254.129:46509
I20260812 21:07:40.971323 12542 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.254.129:46509 every 8 connection(s)
I20260812 21:07:40.971664 12320 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568607354921-12282-0/minicluster-data/ts-0/data/info.pb
Aug 12 21:07:40 dist-test-slave-vpvm krb5kdc[12124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568860, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.129@KRBTEST.COM for kudu/127.11.254.190@KRBTEST.COM
I20260812 21:07:40.979092 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 12320
I20260812 21:07:40.979216 12282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568607354921-12282-0/minicluster-data/ts-0/wal/instance
I20260812 21:07:40.979504 12282 external_mini_cluster.cc:1428] Setting key 12ae849cab18b9d5c0d940df81b4e97f
I20260812 21:07:40.981273 12268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:40.972432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.129:59729 (local address 127.11.254.190:35453)
0812 21:07:40.972545 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0812 21:07:40.972548 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 21:07:40.972698 (+ 150us) server_negotiation.cc:407] Connection header received
0812 21:07:40.973244 (+ 546us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:07:40.973246 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:07:40.973259 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:07:40.973309 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:07:40.974333 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:40.974851 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:40.975856 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:40.976070 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:40.976126 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:07:40.978365 (+ 2239us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:07:40.978366 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:07:40.978366 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:07:40.978374 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:07:40.979860 (+ 1486us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:07:40.980312 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:07:40.980313 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:07:40.980313 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:07:40.980369 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:07:40.980648 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:07:40.980648 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:07:40.980648 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:07:40.980752 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:07:40.980798 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0812 21:07:40.981133 (+ 335us) server_negotiation.cc:299] Negotiation successful
0812 21:07:40.981206 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260812 21:07:40.981539 12547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:40.972400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:35453 (local address 127.11.254.129:59729)
0812 21:07:40.972567 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0812 21:07:40.972613 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 21:07:40.973137 (+ 524us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:07:40.973396 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:07:40.973400 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:07:40.974013 (+ 613us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:07:40.974226 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:07:40.974232 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:07:40.974940 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:07:40.974941 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:07:40.975767 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:07:40.975770 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:07:40.976142 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:07:40.976143 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:07:40.976265 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:07:40.976772 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:07:40.976780 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:07:40.978251 (+ 1471us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:07:40.979940 (+ 1689us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:07:40.979949 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:07:40.979951 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:07:40.980242 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:07:40.980429 (+ 187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:07:40.980430 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:07:40.980430 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:07:40.980588 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:07:40.980802 (+ 214us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:07:40.980808 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:07:40.981005 (+ 197us) client_negotiation.cc:769] Sending connection context
0812 21:07:40.981123 (+ 118us) client_negotiation.cc:240] Negotiation successful
0812 21:07:40.981306 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":81,"threads_started":1}
I20260812 21:07:40.982148 12544 heartbeater.cc:347] Connected to a master server at 127.11.254.190:35453
I20260812 21:07:40.982259 12544 heartbeater.cc:464] Registering TS with master...
I20260812 21:07:40.982455 12544 heartbeater.cc:511] Master 127.11.254.190:35453 requested a full tablet report, sending...
I20260812 21:07:40.982987 12202 ts_manager.cc:194] Registered new tserver with Master: 01ae9577f1e04432b4d5c3391f383e6e (127.11.254.129:46509)
I20260812 21:07:40.983619 12202 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.254.129@KRBTEST.COM'} at 127.11.254.129:59729
WARNING: no policy specified for kudu/127.11.254.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.254.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.254.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.130.keytab.
Entry for principal kudu/127.11.254.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.130.keytab.
I20260812 21:07:41.008666 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568607354921-12282-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568607354921-12282-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568607354921-12282-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568607354921-12282-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.254.130:0
--local_ip_for_outbound_sockets=127.11.254.130
--webserver_interface=127.11.254.130
--webserver_port=0
--tserver_master_addrs=127.11.254.190:35453
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.130.keytab
--principal=kudu/127.11.254.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.254.148:44565
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568607354921-12282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568607354921-12282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568607354921-12282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:07:41.093418 12553 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:07:41.093641 12553 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:07:41.093667 12553 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:07:41.094877 12553 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:07:41.094923 12553 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:07:41.094949 12553 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.254.130
I20260812 21:07:41.096379 12553 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.254.148:44565
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568607354921-12282-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568607354921-12282-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.130.keytab
--principal=kudu/127.11.254.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.254.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568607354921-12282-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.254.130
--webserver_port=0
--tserver_master_addrs=127.11.254.190:35453
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12553
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.254.130
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568607354921-12282-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:07:41.096570 12553 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:07:41.096755 12553 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:07:41.101750 12563 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:07:41.101743 12559 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:07:41.101742 12558 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:07:41.101997 12553 server_base.cc:1034] running on GCE node
I20260812 21:07:41.102212 12553 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:07:41.102432 12553 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:07:41.103593 12553 hybrid_clock.cc:648] HybridClock initialized: now 1786568861103580 us; error 30 us; skew 500 ppm
Aug 12 21:07:41 dist-test-slave-vpvm krb5kdc[12124](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568861, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:07:41.106072 12553 init.cc:377] Logged in from keytab as kudu/127.11.254.130@KRBTEST.COM (short username kudu)
I20260812 21:07:41.106529 12553 webserver.cc:460] Webserver started at http://127.11.254.130:45681/ using document root <none> and password file <none>
I20260812 21:07:41.106758 12553 fs_manager.cc:359] Metadata directory not provided
I20260812 21:07:41.106806 12553 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:07:41.106932 12553 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:07:41.107976 12553 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568607354921-12282-0/minicluster-data/ts-1/data/instance:
uuid: "d5ecaa67ebe74b53955dbb8a15873a67"
format_stamp: "Formatted at 2026-08-12 21:07:41 on dist-test-slave-vpvm"
server_key: "2e1c0c48eb7a0d1cabad7db15f0a430f"
server_key_iv: "aca48755e729e8f0768e068bb6b1e41e"
server_key_version: "encryptionkey@0"
I20260812 21:07:41.108328 12553 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568607354921-12282-0/minicluster-data/ts-1/wal/instance:
uuid: "d5ecaa67ebe74b53955dbb8a15873a67"
format_stamp: "Formatted at 2026-08-12 21:07:41 on dist-test-slave-vpvm"
server_key: "2e1c0c48eb7a0d1cabad7db15f0a430f"
server_key_iv: "aca48755e729e8f0768e068bb6b1e41e"
server_key_version: "encryptionkey@0"
I20260812 21:07:41.109568 12553 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260812 21:07:41.110249 12571 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:07:41.110392 12553 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 21:07:41.110476 12553 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568607354921-12282-0/minicluster-data/ts-1/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568607354921-12282-0/minicluster-data/ts-1/wal
uuid: "d5ecaa67ebe74b53955dbb8a15873a67"
format_stamp: "Formatted at 2026-08-12 21:07:41 on dist-test-slave-vpvm"
server_key: "2e1c0c48eb7a0d1cabad7db15f0a430f"
server_key_iv: "aca48755e729e8f0768e068bb6b1e41e"
server_key_version: "encryptionkey@0"
I20260812 21:07:41.110535 12553 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568607354921-12282-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568607354921-12282-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568607354921-12282-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:07:41.124076 12553 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:07:41.125851 12553 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:07:41.126026 12553 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:07:41.126277 12553 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:07:41.126590 12553 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:07:41.126626 12553 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:07:41.126657 12553 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:07:41.126681 12553 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:07:41.132421 12553 rpc_server.cc:307] RPC server started. Bound to: 127.11.254.130:38697
I20260812 21:07:41.132481 12769 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.254.130:38697 every 8 connection(s)
I20260812 21:07:41.132809 12553 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568607354921-12282-0/minicluster-data/ts-1/data/info.pb
I20260812 21:07:41.134675 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 12553
I20260812 21:07:41.134778 12282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568607354921-12282-0/minicluster-data/ts-1/wal/instance
I20260812 21:07:41.134959 12282 external_mini_cluster.cc:1428] Setting key 04362662c15027368187579b75206925
Aug 12 21:07:41 dist-test-slave-vpvm krb5kdc[12124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568861, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.130@KRBTEST.COM for kudu/127.11.254.190@KRBTEST.COM
I20260812 21:07:41.142799 12268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:41.133553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.130:34957 (local address 127.11.254.190:35453)
0812 21:07:41.133636 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0812 21:07:41.133638 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:07:41.133833 (+ 195us) server_negotiation.cc:407] Connection header received
0812 21:07:41.134365 (+ 532us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:07:41.134366 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:07:41.134376 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:07:41.134429 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:07:41.135578 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:41.136223 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:41.137262 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:41.137539 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:41.137602 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:07:41.139996 (+ 2394us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:07:41.139997 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:07:41.139997 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:07:41.140004 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:07:41.141346 (+ 1342us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:07:41.141830 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:07:41.141831 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:07:41.141831 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:07:41.141881 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:07:41.142168 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:07:41.142168 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:07:41.142168 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:07:41.142271 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:07:41.142321 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0812 21:07:41.142650 (+ 329us) server_negotiation.cc:299] Negotiation successful
0812 21:07:41.142731 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260812 21:07:41.143045 12778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:41.133517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:35453 (local address 127.11.254.130:34957)
0812 21:07:41.133714 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0812 21:07:41.133752 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 21:07:41.134277 (+ 525us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:07:41.134561 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:07:41.134565 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:07:41.135272 (+ 707us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:07:41.135497 (+ 225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:07:41.135502 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:07:41.136321 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:07:41.136321 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:07:41.137174 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:07:41.137177 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:07:41.137619 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:07:41.137620 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:07:41.137725 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:07:41.138256 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:07:41.138265 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:07:41.139907 (+ 1642us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:07:41.141429 (+ 1522us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:07:41.141439 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:07:41.141441 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:07:41.141758 (+ 317us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:07:41.141941 (+ 183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:07:41.141942 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:07:41.141942 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:07:41.142108 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:07:41.142328 (+ 220us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:07:41.142335 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:07:41.142507 (+ 172us) client_negotiation.cc:769] Sending connection context
0812 21:07:41.142626 (+ 119us) client_negotiation.cc:240] Negotiation successful
0812 21:07:41.142808 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":92,"threads_started":1}
I20260812 21:07:41.143554 12773 heartbeater.cc:347] Connected to a master server at 127.11.254.190:35453
I20260812 21:07:41.143653 12773 heartbeater.cc:464] Registering TS with master...
I20260812 21:07:41.143832 12773 heartbeater.cc:511] Master 127.11.254.190:35453 requested a full tablet report, sending...
I20260812 21:07:41.144263 12202 ts_manager.cc:194] Registered new tserver with Master: d5ecaa67ebe74b53955dbb8a15873a67 (127.11.254.130:38697)
WARNING: no policy specified for kudu/127.11.254.131@KRBTEST.COM; defaulting to no policy
I20260812 21:07:41.144771 12202 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.254.130@KRBTEST.COM'} at 127.11.254.130:34957
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.254.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.254.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.131.keytab.
Entry for principal kudu/127.11.254.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.131.keytab.
I20260812 21:07:41.164708 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568607354921-12282-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568607354921-12282-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568607354921-12282-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568607354921-12282-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.254.131:0
--local_ip_for_outbound_sockets=127.11.254.131
--webserver_interface=127.11.254.131
--webserver_port=0
--tserver_master_addrs=127.11.254.190:35453
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.131.keytab
--principal=kudu/127.11.254.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.254.148:44565
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568607354921-12282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568607354921-12282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568607354921-12282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:07:41.247462 12784 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:07:41.247684 12784 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:07:41.247707 12784 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:07:41.248992 12784 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:07:41.249050 12784 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:07:41.249078 12784 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.254.131
I20260812 21:07:41.250638 12784 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.254.148:44565
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568607354921-12282-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568607354921-12282-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.131.keytab
--principal=kudu/127.11.254.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.254.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568607354921-12282-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.254.131
--webserver_port=0
--tserver_master_addrs=127.11.254.190:35453
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12784
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.254.131
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568607354921-12282-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:07:41.250909 12784 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:07:41.251138 12784 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:07:41.256333 12798 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:07:41.256326 12794 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:07:41.256326 12793 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:07:41.256421 12784 server_base.cc:1034] running on GCE node
I20260812 21:07:41.256783 12784 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:07:41.256983 12784 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:07:41.258116 12784 hybrid_clock.cc:648] HybridClock initialized: now 1786568861258104 us; error 30 us; skew 500 ppm
Aug 12 21:07:41 dist-test-slave-vpvm krb5kdc[12124](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568861, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:07:41.260627 12784 init.cc:377] Logged in from keytab as kudu/127.11.254.131@KRBTEST.COM (short username kudu)
I20260812 21:07:41.261086 12784 webserver.cc:460] Webserver started at http://127.11.254.131:38067/ using document root <none> and password file <none>
I20260812 21:07:41.261308 12784 fs_manager.cc:359] Metadata directory not provided
I20260812 21:07:41.261353 12784 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:07:41.261474 12784 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:07:41.262537 12784 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568607354921-12282-0/minicluster-data/ts-2/data/instance:
uuid: "bb01190415284c2fabd13059efea005d"
format_stamp: "Formatted at 2026-08-12 21:07:41 on dist-test-slave-vpvm"
server_key: "c6c65b3056c7a8449986e51230877473"
server_key_iv: "3f9ce44a75fa4f45197bee88242bdbc4"
server_key_version: "encryptionkey@0"
I20260812 21:07:41.262883 12784 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568607354921-12282-0/minicluster-data/ts-2/wal/instance:
uuid: "bb01190415284c2fabd13059efea005d"
format_stamp: "Formatted at 2026-08-12 21:07:41 on dist-test-slave-vpvm"
server_key: "c6c65b3056c7a8449986e51230877473"
server_key_iv: "3f9ce44a75fa4f45197bee88242bdbc4"
server_key_version: "encryptionkey@0"
I20260812 21:07:41.264145 12784 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 21:07:41.264850 12809 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:07:41.265012 12784 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 21:07:41.265103 12784 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568607354921-12282-0/minicluster-data/ts-2/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568607354921-12282-0/minicluster-data/ts-2/wal
uuid: "bb01190415284c2fabd13059efea005d"
format_stamp: "Formatted at 2026-08-12 21:07:41 on dist-test-slave-vpvm"
server_key: "c6c65b3056c7a8449986e51230877473"
server_key_iv: "3f9ce44a75fa4f45197bee88242bdbc4"
server_key_version: "encryptionkey@0"
I20260812 21:07:41.265162 12784 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568607354921-12282-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568607354921-12282-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568607354921-12282-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:07:41.279268 12784 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:07:41.281219 12784 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:07:41.281347 12784 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:07:41.281582 12784 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:07:41.281915 12784 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:07:41.281975 12784 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:07:41.282018 12784 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:07:41.282047 12784 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:07:41.287912 12784 rpc_server.cc:307] RPC server started. Bound to: 127.11.254.131:37307
I20260812 21:07:41.287984 13017 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.254.131:37307 every 8 connection(s)
I20260812 21:07:41.288293 12784 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568607354921-12282-0/minicluster-data/ts-2/data/info.pb
I20260812 21:07:41.290587 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 12784
I20260812 21:07:41.290673 12282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568607354921-12282-0/minicluster-data/ts-2/wal/instance
I20260812 21:07:41.290820 12282 external_mini_cluster.cc:1428] Setting key ecec711a7ced826eb3accf381aad5e59
Aug 12 21:07:41 dist-test-slave-vpvm krb5kdc[12124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568861, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.131@KRBTEST.COM for kudu/127.11.254.190@KRBTEST.COM
I20260812 21:07:41.297806 12268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:41.289039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.131:40219 (local address 127.11.254.190:35453)
0812 21:07:41.289084 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0812 21:07:41.289086 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:07:41.289297 (+ 211us) server_negotiation.cc:407] Connection header received
0812 21:07:41.289851 (+ 554us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:07:41.289852 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:07:41.289865 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:07:41.289909 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:07:41.290970 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:41.291584 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:41.292623 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:41.292822 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:41.292879 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:07:41.295225 (+ 2346us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:07:41.295227 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:07:41.295227 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:07:41.295235 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:07:41.296468 (+ 1233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:07:41.296859 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:07:41.296860 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:07:41.296861 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:07:41.296928 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:07:41.297151 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:07:41.297152 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:07:41.297153 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:07:41.297268 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:07:41.297324 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0812 21:07:41.297597 (+ 273us) server_negotiation.cc:299] Negotiation successful
0812 21:07:41.297683 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 21:07:41.297979 13024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:41.289008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:35453 (local address 127.11.254.131:40219)
0812 21:07:41.289176 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0812 21:07:41.289214 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 21:07:41.289732 (+ 518us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:07:41.290011 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:07:41.290016 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:07:41.290618 (+ 602us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:07:41.290872 (+ 254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:07:41.290879 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:07:41.291673 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:07:41.291674 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:07:41.292542 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:07:41.292545 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:07:41.292891 (+ 346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:07:41.292892 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:07:41.293041 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:07:41.293490 (+ 449us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:07:41.293501 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:07:41.295119 (+ 1618us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:07:41.296550 (+ 1431us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:07:41.296557 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:07:41.296560 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:07:41.296791 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:07:41.296985 (+ 194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:07:41.296985 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:07:41.296986 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:07:41.297096 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:07:41.297321 (+ 225us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:07:41.297328 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:07:41.297474 (+ 146us) client_negotiation.cc:769] Sending connection context
0812 21:07:41.297564 (+ 90us) client_negotiation.cc:240] Negotiation successful
0812 21:07:41.297730 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":88,"threads_started":1}
I20260812 21:07:41.298461 13018 heartbeater.cc:347] Connected to a master server at 127.11.254.190:35453
I20260812 21:07:41.298563 13018 heartbeater.cc:464] Registering TS with master...
I20260812 21:07:41.298719 13018 heartbeater.cc:511] Master 127.11.254.190:35453 requested a full tablet report, sending...
I20260812 21:07:41.299068 12202 ts_manager.cc:194] Registered new tserver with Master: bb01190415284c2fabd13059efea005d (127.11.254.131:37307)
I20260812 21:07:41.299409 12202 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.254.131@KRBTEST.COM'} at 127.11.254.131:40219
I20260812 21:07:41.302505 12282 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 21:07:41.309420 12268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:41.303826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55738 (local address 127.11.254.190:35453)
0812 21:07:41.303881 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0812 21:07:41.303884 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 21:07:41.303973 (+ 89us) server_negotiation.cc:407] Connection header received
0812 21:07:41.304020 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:07:41.304021 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:07:41.304030 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:07:41.304072 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:07:41.304599 (+ 527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:41.305079 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:41.305731 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:41.305913 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:41.305998 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:07:41.307136 (+ 1138us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:07:41.307137 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:07:41.307138 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:07:41.307145 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:07:41.308407 (+ 1262us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:07:41.308764 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:07:41.308765 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:07:41.308765 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:07:41.308823 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:07:41.308969 (+ 146us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:07:41.308969 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:07:41.308969 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:07:41.309113 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:07:41.309190 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0812 21:07:41.309273 (+ 83us) server_negotiation.cc:299] Negotiation successful
0812 21:07:41.309335 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260812 21:07:41.311120 12202 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55738:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 21:07:41.312788 12202 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 21:07:41.319581 13048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:41.315714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51702 (local address 127.11.254.131:37307)
0812 21:07:41.315902 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0812 21:07:41.315906 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 21:07:41.315914 (+ 8us) server_negotiation.cc:407] Connection header received
0812 21:07:41.315953 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:07:41.315955 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:07:41.315970 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:07:41.316078 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:07:41.316265 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:41.317451 (+ 1186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:41.318651 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:41.319181 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:41.319251 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:07:41.319312 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0812 21:07:41.319455 (+ 143us) server_negotiation.cc:299] Negotiation successful
0812 21:07:41.319500 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":89,"threads_started":1}
I20260812 21:07:41.319569 13042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:41.315335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.131:37307 (local address 127.0.0.1:51702)
0812 21:07:41.315491 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0812 21:07:41.315546 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0812 21:07:41.315616 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:07:41.316093 (+ 477us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:07:41.316096 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:07:41.316099 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:07:41.316177 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:07:41.316180 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:07:41.317574 (+ 1394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:07:41.317575 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:07:41.318556 (+ 981us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:07:41.318559 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:07:41.319257 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:07:41.319258 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:07:41.319337 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:07:41.319351 (+ 14us) client_negotiation.cc:769] Sending connection context
0812 21:07:41.319416 (+ 65us) client_negotiation.cc:240] Negotiation successful
0812 21:07:41.319476 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"mutex_wait_us":19,"thread_start_us":48,"threads_started":1}
I20260812 21:07:41.319763 13046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:41.315653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41008 (local address 127.11.254.129:46509)
0812 21:07:41.315843 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0812 21:07:41.315848 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 21:07:41.315894 (+ 46us) server_negotiation.cc:407] Connection header received
0812 21:07:41.315968 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:07:41.315970 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:07:41.315984 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:07:41.316055 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:07:41.316293 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:41.317039 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:41.318441 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:41.319190 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:41.319295 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:07:41.319351 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0812 21:07:41.319623 (+ 272us) server_negotiation.cc:299] Negotiation successful
0812 21:07:41.319673 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":85,"threads_started":1}
I20260812 21:07:41.319983 13044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:41.315517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.129:46509 (local address 127.0.0.1:41008)
0812 21:07:41.315833 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0812 21:07:41.315843 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 21:07:41.315906 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:07:41.316155 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:07:41.316156 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:07:41.316158 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:07:41.316216 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:07:41.316219 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:07:41.317131 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:07:41.317132 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:07:41.318357 (+ 1225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:07:41.318360 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:07:41.319388 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:07:41.319389 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:07:41.319457 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:07:41.319462 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 21:07:41.319505 (+ 43us) client_negotiation.cc:240] Negotiation successful
0812 21:07:41.319548 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":295,"thread_start_us":118,"threads_started":1}
I20260812 21:07:41.320250 13040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:41.315335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.130:38697 (local address 127.0.0.1:52230)
0812 21:07:41.315636 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0812 21:07:41.315644 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0812 21:07:41.315684 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:07:41.316175 (+ 491us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:07:41.316176 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:07:41.316178 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:07:41.316219 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:07:41.316221 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:07:41.317562 (+ 1341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:07:41.317563 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:07:41.318647 (+ 1084us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:07:41.318650 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:07:41.319531 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:07:41.319531 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:07:41.319596 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:07:41.319600 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 21:07:41.319630 (+ 30us) client_negotiation.cc:240] Negotiation successful
0812 21:07:41.320187 (+ 557us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":289,"mutex_wait_us":27,"thread_start_us":87,"threads_started":1}
I20260812 21:07:41.320441 12902 tablet_service.cc:1467] Processing CreateTablet for tablet 7ee482b3b0c14559b63b5c19ddfe1f58 (DEFAULT_TABLE table=test-table [id=0410ab01082f4d0bbb22764018b720a5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:07:41.320645 13043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:41.315445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52230 (local address 127.11.254.130:38697)
0812 21:07:41.315937 (+ 492us) server_negotiation.cc:206] Beginning negotiation
0812 21:07:41.315942 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 21:07:41.315953 (+ 11us) server_negotiation.cc:407] Connection header received
0812 21:07:41.315992 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:07:41.315994 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:07:41.316009 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:07:41.316298 (+ 289us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:07:41.316377 (+ 79us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:41.317445 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:41.318723 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:41.319436 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:41.320405 (+ 969us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:07:41.320477 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0812 21:07:41.320505 (+ 28us) server_negotiation.cc:299] Negotiation successful
0812 21:07:41.320549 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":452,"thread_start_us":106,"threads_started":1}
I20260812 21:07:41.320647 12432 tablet_service.cc:1467] Processing CreateTablet for tablet 7ee482b3b0c14559b63b5c19ddfe1f58 (DEFAULT_TABLE table=test-table [id=0410ab01082f4d0bbb22764018b720a5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:07:41.320832 12902 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7ee482b3b0c14559b63b5c19ddfe1f58. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:07:41.321022 12432 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7ee482b3b0c14559b63b5c19ddfe1f58. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:07:41.321302 12652 tablet_service.cc:1467] Processing CreateTablet for tablet 7ee482b3b0c14559b63b5c19ddfe1f58 (DEFAULT_TABLE table=test-table [id=0410ab01082f4d0bbb22764018b720a5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:07:41.321607 12652 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7ee482b3b0c14559b63b5c19ddfe1f58. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:07:41.323650 13049 tablet_bootstrap.cc:492] T 7ee482b3b0c14559b63b5c19ddfe1f58 P 01ae9577f1e04432b4d5c3391f383e6e: Bootstrap starting.
I20260812 21:07:41.324572 13049 tablet_bootstrap.cc:654] T 7ee482b3b0c14559b63b5c19ddfe1f58 P 01ae9577f1e04432b4d5c3391f383e6e: Neither blocks nor log segments found. Creating new log.
I20260812 21:07:41.324959 13053 tablet_bootstrap.cc:492] T 7ee482b3b0c14559b63b5c19ddfe1f58 P d5ecaa67ebe74b53955dbb8a15873a67: Bootstrap starting.
I20260812 21:07:41.325304 13049 log.cc:824] T 7ee482b3b0c14559b63b5c19ddfe1f58 P 01ae9577f1e04432b4d5c3391f383e6e: Log is configured to *not* fsync() on all Append() calls
I20260812 21:07:41.325650 13053 tablet_bootstrap.cc:654] T 7ee482b3b0c14559b63b5c19ddfe1f58 P d5ecaa67ebe74b53955dbb8a15873a67: Neither blocks nor log segments found. Creating new log.
I20260812 21:07:41.325690 13051 tablet_bootstrap.cc:492] T 7ee482b3b0c14559b63b5c19ddfe1f58 P bb01190415284c2fabd13059efea005d: Bootstrap starting.
I20260812 21:07:41.326171 13049 tablet_bootstrap.cc:492] T 7ee482b3b0c14559b63b5c19ddfe1f58 P 01ae9577f1e04432b4d5c3391f383e6e: No bootstrap required, opened a new log
I20260812 21:07:41.326248 13049 ts_tablet_manager.cc:1397] T 7ee482b3b0c14559b63b5c19ddfe1f58 P 01ae9577f1e04432b4d5c3391f383e6e: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 21:07:41.326248 13053 log.cc:824] T 7ee482b3b0c14559b63b5c19ddfe1f58 P d5ecaa67ebe74b53955dbb8a15873a67: Log is configured to *not* fsync() on all Append() calls
I20260812 21:07:41.327010 13053 tablet_bootstrap.cc:492] T 7ee482b3b0c14559b63b5c19ddfe1f58 P d5ecaa67ebe74b53955dbb8a15873a67: No bootstrap required, opened a new log
I20260812 21:07:41.327080 13053 ts_tablet_manager.cc:1397] T 7ee482b3b0c14559b63b5c19ddfe1f58 P d5ecaa67ebe74b53955dbb8a15873a67: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 21:07:41.328181 13049 raft_consensus.cc:348] T 7ee482b3b0c14559b63b5c19ddfe1f58 P 01ae9577f1e04432b4d5c3391f383e6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb01190415284c2fabd13059efea005d" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 37307 } } peers { permanent_uuid: "d5ecaa67ebe74b53955dbb8a15873a67" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 38697 } } peers { permanent_uuid: "01ae9577f1e04432b4d5c3391f383e6e" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 46509 } }
I20260812 21:07:41.328373 13049 raft_consensus.cc:374] T 7ee482b3b0c14559b63b5c19ddfe1f58 P 01ae9577f1e04432b4d5c3391f383e6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:07:41.328424 13049 raft_consensus.cc:729] T 7ee482b3b0c14559b63b5c19ddfe1f58 P 01ae9577f1e04432b4d5c3391f383e6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01ae9577f1e04432b4d5c3391f383e6e, State: Initialized, Role: FOLLOWER
I20260812 21:07:41.328540 13049 consensus_queue.cc:260] T 7ee482b3b0c14559b63b5c19ddfe1f58 P 01ae9577f1e04432b4d5c3391f383e6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb01190415284c2fabd13059efea005d" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 37307 } } peers { permanent_uuid: "d5ecaa67ebe74b53955dbb8a15873a67" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 38697 } } peers { permanent_uuid: "01ae9577f1e04432b4d5c3391f383e6e" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 46509 } }
I20260812 21:07:41.328836 13049 ts_tablet_manager.cc:1428] T 7ee482b3b0c14559b63b5c19ddfe1f58 P 01ae9577f1e04432b4d5c3391f383e6e: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 21:07:41.328855 12544 heartbeater.cc:503] Master 127.11.254.190:35453 was elected leader, sending a full tablet report...
I20260812 21:07:41.328985 13053 raft_consensus.cc:348] T 7ee482b3b0c14559b63b5c19ddfe1f58 P d5ecaa67ebe74b53955dbb8a15873a67 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb01190415284c2fabd13059efea005d" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 37307 } } peers { permanent_uuid: "d5ecaa67ebe74b53955dbb8a15873a67" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 38697 } } peers { permanent_uuid: "01ae9577f1e04432b4d5c3391f383e6e" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 46509 } }
I20260812 21:07:41.329154 13053 raft_consensus.cc:374] T 7ee482b3b0c14559b63b5c19ddfe1f58 P d5ecaa67ebe74b53955dbb8a15873a67 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:07:41.329187 13053 raft_consensus.cc:729] T 7ee482b3b0c14559b63b5c19ddfe1f58 P d5ecaa67ebe74b53955dbb8a15873a67 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5ecaa67ebe74b53955dbb8a15873a67, State: Initialized, Role: FOLLOWER
I20260812 21:07:41.329329 13053 consensus_queue.cc:260] T 7ee482b3b0c14559b63b5c19ddfe1f58 P d5ecaa67ebe74b53955dbb8a15873a67 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb01190415284c2fabd13059efea005d" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 37307 } } peers { permanent_uuid: "d5ecaa67ebe74b53955dbb8a15873a67" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 38697 } } peers { permanent_uuid: "01ae9577f1e04432b4d5c3391f383e6e" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 46509 } }
I20260812 21:07:41.329545 13051 tablet_bootstrap.cc:654] T 7ee482b3b0c14559b63b5c19ddfe1f58 P bb01190415284c2fabd13059efea005d: Neither blocks nor log segments found. Creating new log.
I20260812 21:07:41.329612 13053 ts_tablet_manager.cc:1428] T 7ee482b3b0c14559b63b5c19ddfe1f58 P d5ecaa67ebe74b53955dbb8a15873a67: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 21:07:41.329682 12773 heartbeater.cc:503] Master 127.11.254.190:35453 was elected leader, sending a full tablet report...
I20260812 21:07:41.330144 13051 log.cc:824] T 7ee482b3b0c14559b63b5c19ddfe1f58 P bb01190415284c2fabd13059efea005d: Log is configured to *not* fsync() on all Append() calls
I20260812 21:07:41.330992 13051 tablet_bootstrap.cc:492] T 7ee482b3b0c14559b63b5c19ddfe1f58 P bb01190415284c2fabd13059efea005d: No bootstrap required, opened a new log
I20260812 21:07:41.331054 13051 ts_tablet_manager.cc:1397] T 7ee482b3b0c14559b63b5c19ddfe1f58 P bb01190415284c2fabd13059efea005d: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260812 21:07:41.332425 13051 raft_consensus.cc:348] T 7ee482b3b0c14559b63b5c19ddfe1f58 P bb01190415284c2fabd13059efea005d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb01190415284c2fabd13059efea005d" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 37307 } } peers { permanent_uuid: "d5ecaa67ebe74b53955dbb8a15873a67" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 38697 } } peers { permanent_uuid: "01ae9577f1e04432b4d5c3391f383e6e" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 46509 } }
I20260812 21:07:41.332576 13051 raft_consensus.cc:374] T 7ee482b3b0c14559b63b5c19ddfe1f58 P bb01190415284c2fabd13059efea005d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:07:41.332599 13051 raft_consensus.cc:729] T 7ee482b3b0c14559b63b5c19ddfe1f58 P bb01190415284c2fabd13059efea005d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb01190415284c2fabd13059efea005d, State: Initialized, Role: FOLLOWER
I20260812 21:07:41.332696 13051 consensus_queue.cc:260] T 7ee482b3b0c14559b63b5c19ddfe1f58 P bb01190415284c2fabd13059efea005d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb01190415284c2fabd13059efea005d" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 37307 } } peers { permanent_uuid: "d5ecaa67ebe74b53955dbb8a15873a67" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 38697 } } peers { permanent_uuid: "01ae9577f1e04432b4d5c3391f383e6e" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 46509 } }
I20260812 21:07:41.332964 13051 ts_tablet_manager.cc:1428] T 7ee482b3b0c14559b63b5c19ddfe1f58 P bb01190415284c2fabd13059efea005d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260812 21:07:41.333003 13018 heartbeater.cc:503] Master 127.11.254.190:35453 was elected leader, sending a full tablet report...
I20260812 21:07:41.368727 13060 raft_consensus.cc:482] T 7ee482b3b0c14559b63b5c19ddfe1f58 P d5ecaa67ebe74b53955dbb8a15873a67 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 21:07:41.368875 13060 raft_consensus.cc:504] T 7ee482b3b0c14559b63b5c19ddfe1f58 P d5ecaa67ebe74b53955dbb8a15873a67 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb01190415284c2fabd13059efea005d" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 37307 } } peers { permanent_uuid: "d5ecaa67ebe74b53955dbb8a15873a67" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 38697 } } peers { permanent_uuid: "01ae9577f1e04432b4d5c3391f383e6e" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 46509 } }
I20260812 21:07:41.369252 13060 leader_election.cc:288] T 7ee482b3b0c14559b63b5c19ddfe1f58 P d5ecaa67ebe74b53955dbb8a15873a67 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bb01190415284c2fabd13059efea005d (127.11.254.131:37307), 01ae9577f1e04432b4d5c3391f383e6e (127.11.254.129:46509)
I20260812 21:07:41.372035 13048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:41.369471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.130:41929 (local address 127.11.254.131:37307)
0812 21:07:41.369520 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0812 21:07:41.369522 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:07:41.369732 (+ 210us) server_negotiation.cc:407] Connection header received
0812 21:07:41.369767 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:07:41.369768 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:07:41.369777 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:07:41.369823 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:07:41.369982 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:41.370443 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:41.371351 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:41.371728 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:41.371793 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:07:41.371801 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 21:07:41.371927 (+ 126us) server_negotiation.cc:299] Negotiation successful
0812 21:07:41.371961 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 21:07:41.372035 13046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:41.369515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.130:53275 (local address 127.11.254.129:46509)
0812 21:07:41.369621 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0812 21:07:41.369623 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:07:41.369634 (+ 11us) server_negotiation.cc:407] Connection header received
0812 21:07:41.369666 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:07:41.369667 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:07:41.369677 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:07:41.369739 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:07:41.369881 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:41.370424 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:41.371340 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:41.371723 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:41.371785 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:07:41.371795 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0812 21:07:41.371932 (+ 137us) server_negotiation.cc:299] Negotiation successful
0812 21:07:41.371969 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260812 21:07:41.372064 12778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:41.369428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.129:46509 (local address 127.11.254.130:53275)
0812 21:07:41.369499 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0812 21:07:41.369510 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 21:07:41.369551 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:07:41.369748 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:07:41.369749 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:07:41.369751 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:07:41.369822 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:07:41.369824 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:07:41.370513 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:07:41.370514 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:07:41.371255 (+ 741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:07:41.371257 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:07:41.371788 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:07:41.371789 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:07:41.371867 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:07:41.371870 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 21:07:41.371913 (+ 43us) client_negotiation.cc:240] Negotiation successful
0812 21:07:41.371955 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52}
I20260812 21:07:41.372081 13064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:41.369428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.131:37307 (local address 127.11.254.130:41929)
0812 21:07:41.369668 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0812 21:07:41.369675 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0812 21:07:41.369718 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:07:41.369830 (+ 112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:07:41.369831 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:07:41.369832 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:07:41.369903 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:07:41.369904 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:07:41.370518 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:07:41.370519 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:07:41.371272 (+ 753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:07:41.371275 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:07:41.371805 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:07:41.371806 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:07:41.371870 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:07:41.371873 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 21:07:41.371915 (+ 42us) client_negotiation.cc:240] Negotiation successful
0812 21:07:41.371955 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":227,"mutex_wait_us":41,"thread_start_us":135,"threads_started":1}
I20260812 21:07:41.372407 12465 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ee482b3b0c14559b63b5c19ddfe1f58" candidate_uuid: "d5ecaa67ebe74b53955dbb8a15873a67" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01ae9577f1e04432b4d5c3391f383e6e" is_pre_election: true
I20260812 21:07:41.372421 12938 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ee482b3b0c14559b63b5c19ddfe1f58" candidate_uuid: "d5ecaa67ebe74b53955dbb8a15873a67" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb01190415284c2fabd13059efea005d" is_pre_election: true
I20260812 21:07:41.372543 12465 raft_consensus.cc:2454] T 7ee482b3b0c14559b63b5c19ddfe1f58 P 01ae9577f1e04432b4d5c3391f383e6e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d5ecaa67ebe74b53955dbb8a15873a67 in term 0.
I20260812 21:07:41.372558 12938 raft_consensus.cc:2454] T 7ee482b3b0c14559b63b5c19ddfe1f58 P bb01190415284c2fabd13059efea005d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d5ecaa67ebe74b53955dbb8a15873a67 in term 0.
I20260812 21:07:41.372746 12574 leader_election.cc:302] T 7ee482b3b0c14559b63b5c19ddfe1f58 P d5ecaa67ebe74b53955dbb8a15873a67 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae9577f1e04432b4d5c3391f383e6e, d5ecaa67ebe74b53955dbb8a15873a67; no voters:
I20260812 21:07:41.372875 13060 raft_consensus.cc:2781] T 7ee482b3b0c14559b63b5c19ddfe1f58 P d5ecaa67ebe74b53955dbb8a15873a67 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 21:07:41.372933 13060 raft_consensus.cc:482] T 7ee482b3b0c14559b63b5c19ddfe1f58 P d5ecaa67ebe74b53955dbb8a15873a67 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 21:07:41.372956 13060 raft_consensus.cc:3037] T 7ee482b3b0c14559b63b5c19ddfe1f58 P d5ecaa67ebe74b53955dbb8a15873a67 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:07:41.373647 13060 raft_consensus.cc:504] T 7ee482b3b0c14559b63b5c19ddfe1f58 P d5ecaa67ebe74b53955dbb8a15873a67 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb01190415284c2fabd13059efea005d" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 37307 } } peers { permanent_uuid: "d5ecaa67ebe74b53955dbb8a15873a67" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 38697 } } peers { permanent_uuid: "01ae9577f1e04432b4d5c3391f383e6e" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 46509 } }
I20260812 21:07:41.373768 13060 leader_election.cc:288] T 7ee482b3b0c14559b63b5c19ddfe1f58 P d5ecaa67ebe74b53955dbb8a15873a67 [CANDIDATE]: Term 1 election: Requested vote from peers bb01190415284c2fabd13059efea005d (127.11.254.131:37307), 01ae9577f1e04432b4d5c3391f383e6e (127.11.254.129:46509)
I20260812 21:07:41.373926 12465 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ee482b3b0c14559b63b5c19ddfe1f58" candidate_uuid: "d5ecaa67ebe74b53955dbb8a15873a67" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01ae9577f1e04432b4d5c3391f383e6e"
I20260812 21:07:41.374008 12938 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ee482b3b0c14559b63b5c19ddfe1f58" candidate_uuid: "d5ecaa67ebe74b53955dbb8a15873a67" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb01190415284c2fabd13059efea005d"
I20260812 21:07:41.374032 12465 raft_consensus.cc:3037] T 7ee482b3b0c14559b63b5c19ddfe1f58 P 01ae9577f1e04432b4d5c3391f383e6e [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:07:41.374089 12938 raft_consensus.cc:3037] T 7ee482b3b0c14559b63b5c19ddfe1f58 P bb01190415284c2fabd13059efea005d [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:07:41.374792 12938 raft_consensus.cc:2454] T 7ee482b3b0c14559b63b5c19ddfe1f58 P bb01190415284c2fabd13059efea005d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d5ecaa67ebe74b53955dbb8a15873a67 in term 1.
I20260812 21:07:41.374802 12465 raft_consensus.cc:2454] T 7ee482b3b0c14559b63b5c19ddfe1f58 P 01ae9577f1e04432b4d5c3391f383e6e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d5ecaa67ebe74b53955dbb8a15873a67 in term 1.
I20260812 21:07:41.374935 12574 leader_election.cc:302] T 7ee482b3b0c14559b63b5c19ddfe1f58 P d5ecaa67ebe74b53955dbb8a15873a67 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ae9577f1e04432b4d5c3391f383e6e, d5ecaa67ebe74b53955dbb8a15873a67; no voters:
I20260812 21:07:41.375037 13060 raft_consensus.cc:2781] T 7ee482b3b0c14559b63b5c19ddfe1f58 P d5ecaa67ebe74b53955dbb8a15873a67 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:07:41.375204 13060 raft_consensus.cc:686] T 7ee482b3b0c14559b63b5c19ddfe1f58 P d5ecaa67ebe74b53955dbb8a15873a67 [term 1 LEADER]: Becoming Leader. State: Replica: d5ecaa67ebe74b53955dbb8a15873a67, State: Running, Role: LEADER
I20260812 21:07:41.375288 13060 consensus_queue.cc:237] T 7ee482b3b0c14559b63b5c19ddfe1f58 P d5ecaa67ebe74b53955dbb8a15873a67 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb01190415284c2fabd13059efea005d" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 37307 } } peers { permanent_uuid: "d5ecaa67ebe74b53955dbb8a15873a67" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 38697 } } peers { permanent_uuid: "01ae9577f1e04432b4d5c3391f383e6e" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 46509 } }
I20260812 21:07:41.376096 12202 catalog_manager.cc:5495] T 7ee482b3b0c14559b63b5c19ddfe1f58 P d5ecaa67ebe74b53955dbb8a15873a67 reported cstate change: term changed from 0 to 1, leader changed from <none> to d5ecaa67ebe74b53955dbb8a15873a67 (127.11.254.130). New cstate: current_term: 1 leader_uuid: "d5ecaa67ebe74b53955dbb8a15873a67" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb01190415284c2fabd13059efea005d" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 37307 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d5ecaa67ebe74b53955dbb8a15873a67" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 38697 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "01ae9577f1e04432b4d5c3391f383e6e" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 46509 } health_report { overall_health: UNKNOWN } } }
I20260812 21:07:41.380874 13043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:41.378744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52232 (local address 127.11.254.130:38697)
0812 21:07:41.378786 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0812 21:07:41.378788 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:07:41.378796 (+ 8us) server_negotiation.cc:407] Connection header received
0812 21:07:41.378857 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:07:41.378857 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:07:41.378875 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:07:41.378934 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 21:07:41.379057 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:41.379493 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:41.380201 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:41.380395 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:41.380447 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:07:41.380588 (+ 141us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 21:07:41.380685 (+ 97us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 21:07:41.380752 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 21:07:41.380789 (+ 37us) server_negotiation.cc:299] Negotiation successful
0812 21:07:41.380809 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 21:07:41.382354 12938 raft_consensus.cc:1261] T 7ee482b3b0c14559b63b5c19ddfe1f58 P bb01190415284c2fabd13059efea005d [term 1 FOLLOWER]: Refusing update from remote peer d5ecaa67ebe74b53955dbb8a15873a67: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 21:07:41.382375 12465 raft_consensus.cc:1261] T 7ee482b3b0c14559b63b5c19ddfe1f58 P 01ae9577f1e04432b4d5c3391f383e6e [term 1 FOLLOWER]: Refusing update from remote peer d5ecaa67ebe74b53955dbb8a15873a67: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 21:07:41.382648 13065 consensus_queue.cc:1035] T 7ee482b3b0c14559b63b5c19ddfe1f58 P d5ecaa67ebe74b53955dbb8a15873a67 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ae9577f1e04432b4d5c3391f383e6e" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 46509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 21:07:41.382771 13060 consensus_queue.cc:1035] T 7ee482b3b0c14559b63b5c19ddfe1f58 P d5ecaa67ebe74b53955dbb8a15873a67 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb01190415284c2fabd13059efea005d" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 37307 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260812 21:07:41.383419 12774 tablet.cc:2369] T 7ee482b3b0c14559b63b5c19ddfe1f58 P d5ecaa67ebe74b53955dbb8a15873a67: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 21:07:41.385167 13075 mvcc.cc:205] Tried to move back new op lower bound from 7317786056220069888 to 7317786056193884160. Current Snapshot: MvccSnapshot[applied={T|T < 7317786056220069888}]
I20260812 21:07:41.385214 13076 mvcc.cc:205] Tried to move back new op lower bound from 7317786056220069888 to 7317786056193884160. Current Snapshot: MvccSnapshot[applied={T|T < 7317786056220069888}]
I20260812 21:07:41.387326 12201 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55738:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 21:07:41.387413 12201 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55738:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 21:07:41.388095 12201 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 168985b0467b438db3e7ca3e87f4662c: Sending DeleteTablet for 3 replicas of tablet 7ee482b3b0c14559b63b5c19ddfe1f58
I20260812 21:07:41.388396 12902 tablet_service.cc:1514] Processing DeleteTablet for tablet 7ee482b3b0c14559b63b5c19ddfe1f58 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:07:41 UTC) from {username='kudu', principal='kudu/127.11.254.190@KRBTEST.COM'} at 127.0.0.1:51702
I20260812 21:07:41.388461 12432 tablet_service.cc:1514] Processing DeleteTablet for tablet 7ee482b3b0c14559b63b5c19ddfe1f58 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:07:41 UTC) from {username='kudu', principal='kudu/127.11.254.190@KRBTEST.COM'} at 127.0.0.1:41008
I20260812 21:07:41.388609 12652 tablet_service.cc:1514] Processing DeleteTablet for tablet 7ee482b3b0c14559b63b5c19ddfe1f58 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:07:41 UTC) from {username='kudu', principal='kudu/127.11.254.190@KRBTEST.COM'} at 127.0.0.1:52230
I20260812 21:07:41.388705 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 12320
I20260812 21:07:41.388795 13092 tablet_replica.cc:330] T 7ee482b3b0c14559b63b5c19ddfe1f58 P d5ecaa67ebe74b53955dbb8a15873a67: stopping tablet replica
I20260812 21:07:41.388876 13092 raft_consensus.cc:2229] T 7ee482b3b0c14559b63b5c19ddfe1f58 P d5ecaa67ebe74b53955dbb8a15873a67 [term 1 LEADER]: Raft consensus shutting down.
I20260812 21:07:41.388953 13092 raft_consensus.cc:2258] T 7ee482b3b0c14559b63b5c19ddfe1f58 P d5ecaa67ebe74b53955dbb8a15873a67 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 21:07:41.389374 13092 ts_tablet_manager.cc:1905] T 7ee482b3b0c14559b63b5c19ddfe1f58 P d5ecaa67ebe74b53955dbb8a15873a67: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 21:07:41.391314 13092 ts_tablet_manager.cc:1918] T 7ee482b3b0c14559b63b5c19ddfe1f58 P d5ecaa67ebe74b53955dbb8a15873a67: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 21:07:41.391811 13090 tablet_replica.cc:330] T 7ee482b3b0c14559b63b5c19ddfe1f58 P bb01190415284c2fabd13059efea005d: stopping tablet replica
I20260812 21:07:41.391933 13090 raft_consensus.cc:2229] T 7ee482b3b0c14559b63b5c19ddfe1f58 P bb01190415284c2fabd13059efea005d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 21:07:41.392030 13090 raft_consensus.cc:2258] T 7ee482b3b0c14559b63b5c19ddfe1f58 P bb01190415284c2fabd13059efea005d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 21:07:41.394223 13092 log.cc:1196] T 7ee482b3b0c14559b63b5c19ddfe1f58 P d5ecaa67ebe74b53955dbb8a15873a67: Deleting WAL directory at /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568607354921-12282-0/minicluster-data/ts-1/wal/wals/7ee482b3b0c14559b63b5c19ddfe1f58
I20260812 21:07:41.394505 13092 ts_tablet_manager.cc:1939] T 7ee482b3b0c14559b63b5c19ddfe1f58 P d5ecaa67ebe74b53955dbb8a15873a67: Deleting consensus metadata
I20260812 21:07:41.400424 12179 catalog_manager.cc:4841] TS d5ecaa67ebe74b53955dbb8a15873a67 (127.11.254.130:38697): tablet 7ee482b3b0c14559b63b5c19ddfe1f58 (table test-table [id=0410ab01082f4d0bbb22764018b720a5]) successfully deleted
W20260812 21:07:41.405684 12179 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.254.129:46509 (error 108)
I20260812 21:07:41.405917 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 12553
I20260812 21:07:41.406157 13090 ts_tablet_manager.cc:1905] T 7ee482b3b0c14559b63b5c19ddfe1f58 P bb01190415284c2fabd13059efea005d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 21:07:41.406340 13040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:41.406220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.129:46509 (local address 127.0.0.1:41024)
0812 21:07:41.406272 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0812 21:07:41.406304 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.129:46509: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260812 21:07:41.407981 12179 catalog_manager.cc:4584] TS 01ae9577f1e04432b4d5c3391f383e6e (127.11.254.129:46509): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7ee482b3b0c14559b63b5c19ddfe1f58: Network error: Client connection negotiation failed: client connection to 127.11.254.129:46509: connect: Connection refused (error 111)
I20260812 21:07:41.408156 13090 ts_tablet_manager.cc:1918] T 7ee482b3b0c14559b63b5c19ddfe1f58 P bb01190415284c2fabd13059efea005d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 21:07:41.408229 13090 log.cc:1196] T 7ee482b3b0c14559b63b5c19ddfe1f58 P bb01190415284c2fabd13059efea005d: Deleting WAL directory at /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786568607354921-12282-0/minicluster-data/ts-2/wal/wals/7ee482b3b0c14559b63b5c19ddfe1f58
I20260812 21:07:41.408528 13090 ts_tablet_manager.cc:1939] T 7ee482b3b0c14559b63b5c19ddfe1f58 P bb01190415284c2fabd13059efea005d: Deleting consensus metadata
I20260812 21:07:41.408974 12183 catalog_manager.cc:4841] TS bb01190415284c2fabd13059efea005d (127.11.254.131:37307): tablet 7ee482b3b0c14559b63b5c19ddfe1f58 (table test-table [id=0410ab01082f4d0bbb22764018b720a5]) successfully deleted
I20260812 21:07:41.415700 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 12784
I20260812 21:07:41.425817 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 12147
2026-08-12T21:07:41Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2089 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-12T21:07:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T21:07:41Z Disabled control of system clock
I20260812 21:07:41.459175 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786568607354921-12282-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786568607354921-12282-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786568607354921-12282-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.254.190:45081
--webserver_interface=127.11.254.190
--webserver_port=0
--builtin_ntp_servers=127.11.254.148:33083
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.254.190:45081
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 21:07:41.542426 13106 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:07:41.542644 13106 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:07:41.542665 13106 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:07:41.543897 13106 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 21:07:41.543942 13106 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:07:41.543960 13106 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:07:41.543974 13106 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 21:07:41.543998 13106 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 21:07:41.545351 13106 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.254.148:33083
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786568607354921-12282-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786568607354921-12282-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.254.190:45081
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.254.190:45081
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.254.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13106
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786568607354921-12282-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:07:41.545591 13106 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:07:41.545883 13106 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:07:41.551142 13121 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:07:41.551113 13117 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:07:41.551121 13118 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:07:41.551208 13106 server_base.cc:1034] running on GCE node
I20260812 21:07:41.551637 13106 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:07:41.551931 13106 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:07:41.553109 13106 hybrid_clock.cc:648] HybridClock initialized: now 1786568861553096 us; error 39 us; skew 500 ppm
I20260812 21:07:41.554566 13106 webserver.cc:460] Webserver started at http://127.11.254.190:32873/ using document root <none> and password file <none>
I20260812 21:07:41.554841 13106 fs_manager.cc:359] Metadata directory not provided
I20260812 21:07:41.554898 13106 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:07:41.555030 13106 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:07:41.556160 13106 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786568607354921-12282-0/minicluster-data/master-0/data/instance:
uuid: "c53674e2465e403eaed972fb7e348d61"
format_stamp: "Formatted at 2026-08-12 21:07:41 on dist-test-slave-vpvm"
server_key: "b1a18951493df2089ce5679b75042a48"
server_key_iv: "9084a87efa09ed958281b6f7a19c7b26"
server_key_version: "encryptionkey@0"
I20260812 21:07:41.556514 13106 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786568607354921-12282-0/minicluster-data/master-0/wal/instance:
uuid: "c53674e2465e403eaed972fb7e348d61"
format_stamp: "Formatted at 2026-08-12 21:07:41 on dist-test-slave-vpvm"
server_key: "b1a18951493df2089ce5679b75042a48"
server_key_iv: "9084a87efa09ed958281b6f7a19c7b26"
server_key_version: "encryptionkey@0"
I20260812 21:07:41.557833 13106 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 21:07:41.558626 13130 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:07:41.558805 13106 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 21:07:41.558892 13106 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786568607354921-12282-0/minicluster-data/master-0/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786568607354921-12282-0/minicluster-data/master-0/wal
uuid: "c53674e2465e403eaed972fb7e348d61"
format_stamp: "Formatted at 2026-08-12 21:07:41 on dist-test-slave-vpvm"
server_key: "b1a18951493df2089ce5679b75042a48"
server_key_iv: "9084a87efa09ed958281b6f7a19c7b26"
server_key_version: "encryptionkey@0"
I20260812 21:07:41.558966 13106 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786568607354921-12282-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786568607354921-12282-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786568607354921-12282-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:07:41.575213 13106 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:07:41.575565 13106 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:07:41.575708 13106 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:07:41.579749 13106 rpc_server.cc:307] RPC server started. Bound to: 127.11.254.190:45081
I20260812 21:07:41.579794 13213 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.254.190:45081 every 8 connection(s)
I20260812 21:07:41.580127 13106 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
I20260812 21:07:41.581111 13215 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:07:41.583442 13215 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c53674e2465e403eaed972fb7e348d61: Bootstrap starting.
I20260812 21:07:41.583878 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 13106
I20260812 21:07:41.583995 12282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786568607354921-12282-0/minicluster-data/master-0/wal/instance
I20260812 21:07:41.584038 13215 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c53674e2465e403eaed972fb7e348d61: Neither blocks nor log segments found. Creating new log.
I20260812 21:07:41.584193 12282 external_mini_cluster.cc:1428] Setting key 9b8ba37b6317d822b6cf4db15f2e0062
I20260812 21:07:41.584507 13215 log.cc:824] T 00000000000000000000000000000000 P c53674e2465e403eaed972fb7e348d61: Log is configured to *not* fsync() on all Append() calls
I20260812 21:07:41.585397 13215 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c53674e2465e403eaed972fb7e348d61: No bootstrap required, opened a new log
I20260812 21:07:41.587234 13215 raft_consensus.cc:348] T 00000000000000000000000000000000 P c53674e2465e403eaed972fb7e348d61 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c53674e2465e403eaed972fb7e348d61" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 45081 } }
I20260812 21:07:41.587375 13215 raft_consensus.cc:374] T 00000000000000000000000000000000 P c53674e2465e403eaed972fb7e348d61 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:07:41.587396 13215 raft_consensus.cc:729] T 00000000000000000000000000000000 P c53674e2465e403eaed972fb7e348d61 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c53674e2465e403eaed972fb7e348d61, State: Initialized, Role: FOLLOWER
I20260812 21:07:41.587517 13215 consensus_queue.cc:260] T 00000000000000000000000000000000 P c53674e2465e403eaed972fb7e348d61 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c53674e2465e403eaed972fb7e348d61" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 45081 } }
I20260812 21:07:41.587589 13215 raft_consensus.cc:388] T 00000000000000000000000000000000 P c53674e2465e403eaed972fb7e348d61 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 21:07:41.587616 13215 raft_consensus.cc:482] T 00000000000000000000000000000000 P c53674e2465e403eaed972fb7e348d61 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 21:07:41.587666 13215 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c53674e2465e403eaed972fb7e348d61 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:07:41.588202 13215 raft_consensus.cc:504] T 00000000000000000000000000000000 P c53674e2465e403eaed972fb7e348d61 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c53674e2465e403eaed972fb7e348d61" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 45081 } }
I20260812 21:07:41.588301 13215 leader_election.cc:302] T 00000000000000000000000000000000 P c53674e2465e403eaed972fb7e348d61 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c53674e2465e403eaed972fb7e348d61; no voters:
I20260812 21:07:41.588485 13215 leader_election.cc:288] T 00000000000000000000000000000000 P c53674e2465e403eaed972fb7e348d61 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 21:07:41.588542 13221 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c53674e2465e403eaed972fb7e348d61 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:07:41.588762 13215 sys_catalog.cc:564] T 00000000000000000000000000000000 P c53674e2465e403eaed972fb7e348d61 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 21:07:41.588878 13221 raft_consensus.cc:686] T 00000000000000000000000000000000 P c53674e2465e403eaed972fb7e348d61 [term 1 LEADER]: Becoming Leader. State: Replica: c53674e2465e403eaed972fb7e348d61, State: Running, Role: LEADER
I20260812 21:07:41.588989 13221 consensus_queue.cc:237] T 00000000000000000000000000000000 P c53674e2465e403eaed972fb7e348d61 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c53674e2465e403eaed972fb7e348d61" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 45081 } }
I20260812 21:07:41.589318 13225 sys_catalog.cc:455] T 00000000000000000000000000000000 P c53674e2465e403eaed972fb7e348d61 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c53674e2465e403eaed972fb7e348d61. Latest consensus state: current_term: 1 leader_uuid: "c53674e2465e403eaed972fb7e348d61" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c53674e2465e403eaed972fb7e348d61" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 45081 } } }
I20260812 21:07:41.589401 13225 sys_catalog.cc:458] T 00000000000000000000000000000000 P c53674e2465e403eaed972fb7e348d61 [sys.catalog]: This master's current role is: LEADER
I20260812 21:07:41.589737 13240 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 21:07:41.589787 13219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:41.584715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34008 (local address 127.11.254.190:45081)
0812 21:07:41.584917 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0812 21:07:41.584926 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 21:07:41.584952 (+ 26us) server_negotiation.cc:407] Connection header received
0812 21:07:41.585381 (+ 429us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:07:41.585387 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:07:41.585472 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:07:41.585604 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:07:41.585818 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:41.586793 (+ 975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:41.587605 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:41.588091 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:41.588200 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:07:41.588895 (+ 695us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:07:41.588900 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:07:41.588900 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 21:07:41.589103 (+ 203us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:07:41.589145 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 21:07:41.589159 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:07:41.589226 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 21:07:41.589473 (+ 247us) server_negotiation.cc:299] Negotiation successful
0812 21:07:41.589574 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":70,"threads_started":1}
I20260812 21:07:41.590435 13240 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 21:07:41.590559 13224 sys_catalog.cc:455] T 00000000000000000000000000000000 P c53674e2465e403eaed972fb7e348d61 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c53674e2465e403eaed972fb7e348d61" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c53674e2465e403eaed972fb7e348d61" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 45081 } } }
I20260812 21:07:41.590644 13224 sys_catalog.cc:458] T 00000000000000000000000000000000 P c53674e2465e403eaed972fb7e348d61 [sys.catalog]: This master's current role is: LEADER
I20260812 21:07:41.591775 13240 catalog_manager.cc:1347] Generated new cluster ID: 136a17b85d654e9bae2d1fcd0b91a320
I20260812 21:07:41.591827 13240 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 21:07:41.601070 13240 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 21:07:41.601900 13240 catalog_manager.cc:1504] Loading token signing keys...
I20260812 21:07:41.604732 13240 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c53674e2465e403eaed972fb7e348d61: Generated new TSK 0
I20260812 21:07:41.604944 13240 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 21:07:41.607034 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786568607354921-12282-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786568607354921-12282-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786568607354921-12282-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786568607354921-12282-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.254.129:0
--local_ip_for_outbound_sockets=127.11.254.129
--webserver_interface=127.11.254.129
--webserver_port=0
--tserver_master_addrs=127.11.254.190:45081
--builtin_ntp_servers=127.11.254.148:33083
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 21:07:41.692476 13254 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:07:41.692698 13254 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:07:41.692720 13254 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:07:41.693992 13254 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:07:41.694039 13254 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:07:41.694065 13254 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.254.129
I20260812 21:07:41.695472 13254 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.254.148:33083
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786568607354921-12282-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786568607354921-12282-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.254.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786568607354921-12282-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.254.129
--webserver_port=0
--tserver_master_addrs=127.11.254.190:45081
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13254
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.254.129
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786568607354921-12282-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:07:41.695710 13254 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:07:41.696012 13254 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:07:41.702234 13263 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:07:41.702227 13262 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:07:41.702389 13254 server_base.cc:1034] running on GCE node
W20260812 21:07:41.702227 13266 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:07:41.702802 13254 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:07:41.703088 13254 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:07:41.704280 13254 hybrid_clock.cc:648] HybridClock initialized: now 1786568861704257 us; error 38 us; skew 500 ppm
I20260812 21:07:41.705550 13254 webserver.cc:460] Webserver started at http://127.11.254.129:41919/ using document root <none> and password file <none>
I20260812 21:07:41.705776 13254 fs_manager.cc:359] Metadata directory not provided
I20260812 21:07:41.705821 13254 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:07:41.706002 13254 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:07:41.707135 13254 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786568607354921-12282-0/minicluster-data/ts-0/data/instance:
uuid: "7873e78bc42342c1994bba9de68b2578"
format_stamp: "Formatted at 2026-08-12 21:07:41 on dist-test-slave-vpvm"
server_key: "549a25bf3bf75067d30cb8f5be3e3432"
server_key_iv: "8258b1c3cec4ba39b7c63b7124d040ca"
server_key_version: "encryptionkey@0"
I20260812 21:07:41.707492 13254 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786568607354921-12282-0/minicluster-data/ts-0/wal/instance:
uuid: "7873e78bc42342c1994bba9de68b2578"
format_stamp: "Formatted at 2026-08-12 21:07:41 on dist-test-slave-vpvm"
server_key: "549a25bf3bf75067d30cb8f5be3e3432"
server_key_iv: "8258b1c3cec4ba39b7c63b7124d040ca"
server_key_version: "encryptionkey@0"
I20260812 21:07:41.708827 13254 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 21:07:41.709676 13275 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:07:41.709834 13254 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 21:07:41.709913 13254 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786568607354921-12282-0/minicluster-data/ts-0/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786568607354921-12282-0/minicluster-data/ts-0/wal
uuid: "7873e78bc42342c1994bba9de68b2578"
format_stamp: "Formatted at 2026-08-12 21:07:41 on dist-test-slave-vpvm"
server_key: "549a25bf3bf75067d30cb8f5be3e3432"
server_key_iv: "8258b1c3cec4ba39b7c63b7124d040ca"
server_key_version: "encryptionkey@0"
I20260812 21:07:41.710011 13254 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786568607354921-12282-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786568607354921-12282-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786568607354921-12282-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:07:41.762416 13254 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:07:41.762753 13254 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:07:41.762905 13254 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:07:41.763204 13254 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:07:41.763574 13254 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:07:41.763612 13254 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:07:41.763634 13254 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:07:41.763645 13254 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:07:41.769079 13254 rpc_server.cc:307] RPC server started. Bound to: 127.11.254.129:36327
I20260812 21:07:41.769162 13481 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.254.129:36327 every 8 connection(s)
I20260812 21:07:41.769465 13254 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786568607354921-12282-0/minicluster-data/ts-0/data/info.pb
I20260812 21:07:41.773485 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 13254
I20260812 21:07:41.773597 12282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786568607354921-12282-0/minicluster-data/ts-0/wal/instance
I20260812 21:07:41.773816 12282 external_mini_cluster.cc:1428] Setting key 7eb00f9511dd7a4df92692df94141e18
I20260812 21:07:41.774946 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786568607354921-12282-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786568607354921-12282-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786568607354921-12282-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786568607354921-12282-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.254.130:0
--local_ip_for_outbound_sockets=127.11.254.130
--webserver_interface=127.11.254.130
--webserver_port=0
--tserver_master_addrs=127.11.254.190:45081
--builtin_ntp_servers=127.11.254.148:33083
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 21:07:41.775467 13219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:41.770269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.129:58155 (local address 127.11.254.190:45081)
0812 21:07:41.770341 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0812 21:07:41.770344 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 21:07:41.770691 (+ 347us) server_negotiation.cc:407] Connection header received
0812 21:07:41.771202 (+ 511us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:07:41.771203 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:07:41.771215 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:07:41.771276 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:07:41.772036 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:41.772578 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:41.773933 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:41.774204 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:41.774276 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:07:41.775177 (+ 901us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:07:41.775178 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:07:41.775179 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 21:07:41.775184 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:07:41.775202 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 21:07:41.775207 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:07:41.775237 (+ 30us) server_negotiation.cc:1035] Waiting for connection context
0812 21:07:41.775374 (+ 137us) server_negotiation.cc:299] Negotiation successful
0812 21:07:41.775404 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260812 21:07:41.775727 13484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:41.770264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:45081 (local address 127.11.254.129:58155)
0812 21:07:41.770528 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0812 21:07:41.770578 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0812 21:07:41.771089 (+ 511us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:07:41.771346 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:07:41.771350 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:07:41.771360 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:07:41.771915 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:07:41.771920 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:07:41.772711 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:07:41.772712 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:07:41.773807 (+ 1095us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:07:41.773810 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:07:41.774286 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:07:41.774287 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:07:41.774454 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:07:41.774987 (+ 533us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 21:07:41.774988 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:07:41.775031 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 21:07:41.775065 (+ 34us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:07:41.775279 (+ 214us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:07:41.775287 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:07:41.775290 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 21:07:41.775356 (+ 66us) client_negotiation.cc:240] Negotiation successful
0812 21:07:41.775484 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":109,"threads_started":1}
I20260812 21:07:41.776316 13482 heartbeater.cc:347] Connected to a master server at 127.11.254.190:45081
I20260812 21:07:41.776415 13482 heartbeater.cc:464] Registering TS with master...
I20260812 21:07:41.776605 13482 heartbeater.cc:511] Master 127.11.254.190:45081 requested a full tablet report, sending...
I20260812 21:07:41.777112 13154 ts_manager.cc:194] Registered new tserver with Master: 7873e78bc42342c1994bba9de68b2578 (127.11.254.129:36327)
I20260812 21:07:41.777719 13154 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.254.129:58155
W20260812 21:07:41.858807 13486 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:07:41.859050 13486 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:07:41.859076 13486 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:07:41.860301 13486 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:07:41.860354 13486 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:07:41.860390 13486 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.254.130
I20260812 21:07:41.861814 13486 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.254.148:33083
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786568607354921-12282-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786568607354921-12282-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.254.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786568607354921-12282-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.254.130
--webserver_port=0
--tserver_master_addrs=127.11.254.190:45081
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13486
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.254.130
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786568607354921-12282-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:07:41.862133 13486 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:07:41.862470 13486 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:07:41.867637 13502 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:07:41.867667 13486 server_base.cc:1034] running on GCE node
W20260812 21:07:41.867650 13498 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:07:41.867831 13494 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:07:41.868091 13486 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:07:41.868330 13486 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:07:41.869468 13486 hybrid_clock.cc:648] HybridClock initialized: now 1786568861869457 us; error 31 us; skew 500 ppm
I20260812 21:07:41.870776 13486 webserver.cc:460] Webserver started at http://127.11.254.130:46355/ using document root <none> and password file <none>
I20260812 21:07:41.871016 13486 fs_manager.cc:359] Metadata directory not provided
I20260812 21:07:41.871080 13486 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:07:41.871208 13486 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:07:41.872279 13486 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786568607354921-12282-0/minicluster-data/ts-1/data/instance:
uuid: "d0d15142d5fb46958bbd6808fcd22186"
format_stamp: "Formatted at 2026-08-12 21:07:41 on dist-test-slave-vpvm"
server_key: "c02d0576d343ab432f8d22a222192d24"
server_key_iv: "eb81a399ada45f51536d10ddcc949854"
server_key_version: "encryptionkey@0"
I20260812 21:07:41.872637 13486 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786568607354921-12282-0/minicluster-data/ts-1/wal/instance:
uuid: "d0d15142d5fb46958bbd6808fcd22186"
format_stamp: "Formatted at 2026-08-12 21:07:41 on dist-test-slave-vpvm"
server_key: "c02d0576d343ab432f8d22a222192d24"
server_key_iv: "eb81a399ada45f51536d10ddcc949854"
server_key_version: "encryptionkey@0"
I20260812 21:07:41.875304 13486 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.004s
I20260812 21:07:41.876088 13513 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:07:41.876271 13486 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 21:07:41.876358 13486 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786568607354921-12282-0/minicluster-data/ts-1/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786568607354921-12282-0/minicluster-data/ts-1/wal
uuid: "d0d15142d5fb46958bbd6808fcd22186"
format_stamp: "Formatted at 2026-08-12 21:07:41 on dist-test-slave-vpvm"
server_key: "c02d0576d343ab432f8d22a222192d24"
server_key_iv: "eb81a399ada45f51536d10ddcc949854"
server_key_version: "encryptionkey@0"
I20260812 21:07:41.876425 13486 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786568607354921-12282-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786568607354921-12282-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786568607354921-12282-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:07:41.886749 13486 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:07:41.887094 13486 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:07:41.887235 13486 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:07:41.887459 13486 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:07:41.887796 13486 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:07:41.887830 13486 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:07:41.887861 13486 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:07:41.887884 13486 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:07:41.893733 13486 rpc_server.cc:307] RPC server started. Bound to: 127.11.254.130:37329
I20260812 21:07:41.893797 13719 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.254.130:37329 every 8 connection(s)
I20260812 21:07:41.894217 13486 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786568607354921-12282-0/minicluster-data/ts-1/data/info.pb
I20260812 21:07:41.899580 13219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:41.895007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.130:50425 (local address 127.11.254.190:45081)
0812 21:07:41.895093 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0812 21:07:41.895095 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:07:41.895305 (+ 210us) server_negotiation.cc:407] Connection header received
0812 21:07:41.895829 (+ 524us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:07:41.895830 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:07:41.895841 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:07:41.895892 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:07:41.896569 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:41.897065 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:41.898311 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:41.898505 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:41.898565 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:07:41.899331 (+ 766us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:07:41.899333 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:07:41.899333 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 21:07:41.899337 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:07:41.899353 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 21:07:41.899357 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:07:41.899395 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0812 21:07:41.899457 (+ 62us) server_negotiation.cc:299] Negotiation successful
0812 21:07:41.899484 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260812 21:07:41.899765 13725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:41.894981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:45081 (local address 127.11.254.130:50425)
0812 21:07:41.895183 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0812 21:07:41.895220 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0812 21:07:41.895745 (+ 525us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:07:41.895952 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:07:41.895956 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:07:41.895964 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:07:41.896502 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:07:41.896507 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:07:41.897168 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:07:41.897169 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:07:41.898229 (+ 1060us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:07:41.898232 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:07:41.898582 (+ 350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:07:41.898583 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:07:41.898700 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:07:41.899199 (+ 499us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 21:07:41.899199 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:07:41.899230 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 21:07:41.899255 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:07:41.899401 (+ 146us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:07:41.899404 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:07:41.899407 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 21:07:41.899452 (+ 45us) client_negotiation.cc:240] Negotiation successful
0812 21:07:41.899546 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":98,"threads_started":1}
I20260812 21:07:41.900245 13720 heartbeater.cc:347] Connected to a master server at 127.11.254.190:45081
I20260812 21:07:41.900342 13720 heartbeater.cc:464] Registering TS with master...
I20260812 21:07:41.900559 13720 heartbeater.cc:511] Master 127.11.254.190:45081 requested a full tablet report, sending...
I20260812 21:07:41.900910 13154 ts_manager.cc:194] Registered new tserver with Master: d0d15142d5fb46958bbd6808fcd22186 (127.11.254.130:37329)
I20260812 21:07:41.901031 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 13486
I20260812 21:07:41.901142 12282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786568607354921-12282-0/minicluster-data/ts-1/wal/instance
I20260812 21:07:41.901338 12282 external_mini_cluster.cc:1428] Setting key ea072f5cf969816905a708880833070e
I20260812 21:07:41.901340 13154 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.254.130:50425
I20260812 21:07:41.902240 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786568607354921-12282-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786568607354921-12282-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786568607354921-12282-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786568607354921-12282-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.254.131:0
--local_ip_for_outbound_sockets=127.11.254.131
--webserver_interface=127.11.254.131
--webserver_port=0
--tserver_master_addrs=127.11.254.190:45081
--builtin_ntp_servers=127.11.254.148:33083
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 21:07:41.984359 13727 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:07:41.984643 13727 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:07:41.984668 13727 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:07:41.985895 13727 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:07:41.985945 13727 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:07:41.986008 13727 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.254.131
I20260812 21:07:41.987408 13727 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.254.148:33083
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786568607354921-12282-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786568607354921-12282-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.254.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786568607354921-12282-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.254.131
--webserver_port=0
--tserver_master_addrs=127.11.254.190:45081
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13727
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.254.131
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786568607354921-12282-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:07:41.987613 13727 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:07:41.987882 13727 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:07:41.993278 13737 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:07:41.993394 13740 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:07:41.993278 13734 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:07:41.993476 13727 server_base.cc:1034] running on GCE node
I20260812 21:07:41.993763 13727 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:07:41.994048 13727 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:07:41.995215 13727 hybrid_clock.cc:648] HybridClock initialized: now 1786568861995191 us; error 34 us; skew 500 ppm
I20260812 21:07:41.996484 13727 webserver.cc:460] Webserver started at http://127.11.254.131:35275/ using document root <none> and password file <none>
I20260812 21:07:41.996779 13727 fs_manager.cc:359] Metadata directory not provided
I20260812 21:07:41.996831 13727 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:07:41.996963 13727 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:07:41.998054 13727 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786568607354921-12282-0/minicluster-data/ts-2/data/instance:
uuid: "ec9b702bb1e64be2aaa0fa97a8b649eb"
format_stamp: "Formatted at 2026-08-12 21:07:41 on dist-test-slave-vpvm"
server_key: "d106bc1cdabdee90a2938f8388d72759"
server_key_iv: "b2aeccdba8b323713db1c19cc4ac46d7"
server_key_version: "encryptionkey@0"
I20260812 21:07:41.998396 13727 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786568607354921-12282-0/minicluster-data/ts-2/wal/instance:
uuid: "ec9b702bb1e64be2aaa0fa97a8b649eb"
format_stamp: "Formatted at 2026-08-12 21:07:41 on dist-test-slave-vpvm"
server_key: "d106bc1cdabdee90a2938f8388d72759"
server_key_iv: "b2aeccdba8b323713db1c19cc4ac46d7"
server_key_version: "encryptionkey@0"
I20260812 21:07:41.999675 13727 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 21:07:42.000409 13746 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:07:42.000582 13727 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 21:07:42.000667 13727 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786568607354921-12282-0/minicluster-data/ts-2/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786568607354921-12282-0/minicluster-data/ts-2/wal
uuid: "ec9b702bb1e64be2aaa0fa97a8b649eb"
format_stamp: "Formatted at 2026-08-12 21:07:41 on dist-test-slave-vpvm"
server_key: "d106bc1cdabdee90a2938f8388d72759"
server_key_iv: "b2aeccdba8b323713db1c19cc4ac46d7"
server_key_version: "encryptionkey@0"
I20260812 21:07:42.000731 13727 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786568607354921-12282-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786568607354921-12282-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786568607354921-12282-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:07:42.010381 13727 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:07:42.010716 13727 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:07:42.010855 13727 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:07:42.011080 13727 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 21:07:42.011418 13727 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 21:07:42.011452 13727 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 21:07:42.011482 13727 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 21:07:42.011505 13727 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 21:07:42.017879 13727 rpc_server.cc:307] RPC server started. Bound to: 127.11.254.131:39145
I20260812 21:07:42.018081 13946 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.254.131:39145 every 8 connection(s)
I20260812 21:07:42.018379 13727 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786568607354921-12282-0/minicluster-data/ts-2/data/info.pb
I20260812 21:07:42.028556 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 13727
I20260812 21:07:42.028692 12282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786568607354921-12282-0/minicluster-data/ts-2/wal/instance
I20260812 21:07:42.028927 12282 external_mini_cluster.cc:1428] Setting key fb2c9636f097c4ba88b9a5a9a2fd0d73
I20260812 21:07:42.030638 13219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:42.025227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.131:48975 (local address 127.11.254.190:45081)
0812 21:07:42.025293 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0812 21:07:42.025295 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:07:42.025551 (+ 256us) server_negotiation.cc:407] Connection header received
0812 21:07:42.026268 (+ 717us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:07:42.026270 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:07:42.026283 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:07:42.026334 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:07:42.027225 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:42.027848 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:42.029147 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:42.029397 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:42.029455 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:07:42.030368 (+ 913us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:07:42.030369 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:07:42.030370 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 21:07:42.030375 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:07:42.030391 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 21:07:42.030395 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:07:42.030442 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0812 21:07:42.030524 (+ 82us) server_negotiation.cc:299] Negotiation successful
0812 21:07:42.030555 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260812 21:07:42.030848 13950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:42.023608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:45081 (local address 127.11.254.131:48975)
0812 21:07:42.025388 (+ 1780us) negotiation.cc:107] Waiting for socket to connect
0812 21:07:42.025446 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0812 21:07:42.026151 (+ 705us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:07:42.026458 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:07:42.026462 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:07:42.026474 (+ 12us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:07:42.027122 (+ 648us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:07:42.027130 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:07:42.027940 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:07:42.027941 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:07:42.029035 (+ 1094us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:07:42.029038 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:07:42.029485 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:07:42.029486 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:07:42.029644 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:07:42.030228 (+ 584us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 21:07:42.030229 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:07:42.030260 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 21:07:42.030286 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:07:42.030449 (+ 163us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:07:42.030455 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:07:42.030458 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 21:07:42.030519 (+ 61us) client_negotiation.cc:240] Negotiation successful
0812 21:07:42.030634 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1707,"thread_start_us":1600,"threads_started":1}
I20260812 21:07:42.031230 13947 heartbeater.cc:347] Connected to a master server at 127.11.254.190:45081
I20260812 21:07:42.031316 13947 heartbeater.cc:464] Registering TS with master...
I20260812 21:07:42.031502 13947 heartbeater.cc:511] Master 127.11.254.190:45081 requested a full tablet report, sending...
I20260812 21:07:42.031900 13154 ts_manager.cc:194] Registered new tserver with Master: ec9b702bb1e64be2aaa0fa97a8b649eb (127.11.254.131:39145)
I20260812 21:07:42.032295 13154 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.254.131:48975
I20260812 21:07:42.040853 12282 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 21:07:42.045092 13219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:42.042305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34014 (local address 127.11.254.190:45081)
0812 21:07:42.042364 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0812 21:07:42.042366 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:07:42.042476 (+ 110us) server_negotiation.cc:407] Connection header received
0812 21:07:42.042523 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:07:42.042524 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:07:42.042534 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:07:42.042585 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:07:42.042729 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:42.043250 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:42.044029 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:42.044218 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:42.044273 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:07:42.044894 (+ 621us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:07:42.044902 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:07:42.044902 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 21:07:42.044906 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:07:42.044921 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 21:07:42.044924 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:07:42.044966 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0812 21:07:42.045006 (+ 40us) server_negotiation.cc:299] Negotiation successful
0812 21:07:42.045028 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260812 21:07:42.046597 13154 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34014:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 21:07:42.048167 13154 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 21:07:42.054579 13967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:42.050617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.131:39145 (local address 127.0.0.1:49638)
0812 21:07:42.050751 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0812 21:07:42.050762 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 21:07:42.050806 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:07:42.051344 (+ 538us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:07:42.051345 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:07:42.051347 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:07:42.051388 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:07:42.051389 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:07:42.052525 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:07:42.052526 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:07:42.053443 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:07:42.053446 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:07:42.054260 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:07:42.054261 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:07:42.054340 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:07:42.054354 (+ 14us) client_negotiation.cc:769] Sending connection context
0812 21:07:42.054438 (+ 84us) client_negotiation.cc:240] Negotiation successful
0812 21:07:42.054490 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"mutex_wait_us":1,"thread_start_us":74,"threads_started":1}
I20260812 21:07:42.054589 13971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:42.050990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49638 (local address 127.11.254.131:39145)
0812 21:07:42.051182 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0812 21:07:42.051186 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 21:07:42.051194 (+ 8us) server_negotiation.cc:407] Connection header received
0812 21:07:42.051229 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:07:42.051231 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:07:42.051246 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:07:42.051448 (+ 202us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:07:42.051502 (+ 54us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:42.052457 (+ 955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:42.053517 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:42.054179 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:42.054253 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:07:42.054338 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0812 21:07:42.054457 (+ 119us) server_negotiation.cc:299] Negotiation successful
0812 21:07:42.054494 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":88,"threads_started":1}
I20260812 21:07:42.055110 13960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:42.050339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.129:36327 (local address 127.0.0.1:58834)
0812 21:07:42.050511 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0812 21:07:42.050555 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 21:07:42.050618 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:07:42.050893 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:07:42.050897 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:07:42.050899 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:07:42.050970 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:07:42.050972 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:07:42.052294 (+ 1322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:07:42.052295 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:07:42.053676 (+ 1381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:07:42.053679 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:07:42.054856 (+ 1177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:07:42.054858 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:07:42.054932 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:07:42.054936 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 21:07:42.054994 (+ 58us) client_negotiation.cc:240] Negotiation successful
0812 21:07:42.055038 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"mutex_wait_us":54,"thread_start_us":123,"threads_started":1}
I20260812 21:07:42.055128 13969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:42.050771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.130:37329 (local address 127.0.0.1:40286)
0812 21:07:42.051022 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0812 21:07:42.051031 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 21:07:42.051079 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:07:42.051439 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:07:42.051439 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:07:42.051441 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:07:42.051480 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:07:42.051481 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:07:42.052316 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:07:42.052316 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:07:42.053676 (+ 1360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:07:42.053679 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:07:42.054892 (+ 1213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:07:42.054893 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:07:42.054944 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:07:42.054948 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 21:07:42.054988 (+ 40us) client_negotiation.cc:240] Negotiation successful
0812 21:07:42.055040 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":233,"mutex_wait_us":65,"thread_start_us":70,"threads_started":1}
I20260812 21:07:42.055337 13970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:42.050920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40286 (local address 127.11.254.130:37329)
0812 21:07:42.051242 (+ 322us) server_negotiation.cc:206] Beginning negotiation
0812 21:07:42.051246 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 21:07:42.051256 (+ 10us) server_negotiation.cc:407] Connection header received
0812 21:07:42.051293 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:07:42.051295 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:07:42.051310 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:07:42.051376 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:07:42.051546 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:42.052231 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:42.053764 (+ 1533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:42.054516 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:42.054580 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:07:42.054662 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0812 21:07:42.055218 (+ 556us) server_negotiation.cc:299] Negotiation successful
0812 21:07:42.055260 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":283,"thread_start_us":103,"threads_started":1}
I20260812 21:07:42.055465 13831 tablet_service.cc:1467] Processing CreateTablet for tablet 780784f9ab7d4599a6a63b616d980c18 (DEFAULT_TABLE table=test-table [id=46062005ed2e42f5a5a4c88c4a6841c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:07:42.055851 13831 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 780784f9ab7d4599a6a63b616d980c18. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:07:42.055910 13590 tablet_service.cc:1467] Processing CreateTablet for tablet 780784f9ab7d4599a6a63b616d980c18 (DEFAULT_TABLE table=test-table [id=46062005ed2e42f5a5a4c88c4a6841c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:07:42.056219 13590 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 780784f9ab7d4599a6a63b616d980c18. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:07:42.058223 13963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:42.050445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58834 (local address 127.11.254.129:36327)
0812 21:07:42.050642 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0812 21:07:42.050647 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 21:07:42.050661 (+ 14us) server_negotiation.cc:407] Connection header received
0812 21:07:42.050701 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:07:42.050703 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:07:42.050720 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:07:42.050809 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:07:42.051067 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:42.052209 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:42.054020 (+ 1811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:42.054662 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:42.054731 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:07:42.054817 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0812 21:07:42.058091 (+ 3274us) server_negotiation.cc:299] Negotiation successful
0812 21:07:42.058141 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":109,"threads_started":1}
I20260812 21:07:42.058802 13973 tablet_bootstrap.cc:492] T 780784f9ab7d4599a6a63b616d980c18 P d0d15142d5fb46958bbd6808fcd22186: Bootstrap starting.
I20260812 21:07:42.059118 13359 tablet_service.cc:1467] Processing CreateTablet for tablet 780784f9ab7d4599a6a63b616d980c18 (DEFAULT_TABLE table=test-table [id=46062005ed2e42f5a5a4c88c4a6841c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 21:07:42.059378 13973 tablet_bootstrap.cc:654] T 780784f9ab7d4599a6a63b616d980c18 P d0d15142d5fb46958bbd6808fcd22186: Neither blocks nor log segments found. Creating new log.
I20260812 21:07:42.059396 13359 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 780784f9ab7d4599a6a63b616d980c18. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:07:42.059649 13973 log.cc:824] T 780784f9ab7d4599a6a63b616d980c18 P d0d15142d5fb46958bbd6808fcd22186: Log is configured to *not* fsync() on all Append() calls
I20260812 21:07:42.060272 13973 tablet_bootstrap.cc:492] T 780784f9ab7d4599a6a63b616d980c18 P d0d15142d5fb46958bbd6808fcd22186: No bootstrap required, opened a new log
I20260812 21:07:42.060326 13973 ts_tablet_manager.cc:1397] T 780784f9ab7d4599a6a63b616d980c18 P d0d15142d5fb46958bbd6808fcd22186: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 21:07:42.061965 13975 tablet_bootstrap.cc:492] T 780784f9ab7d4599a6a63b616d980c18 P 7873e78bc42342c1994bba9de68b2578: Bootstrap starting.
I20260812 21:07:42.062085 13972 tablet_bootstrap.cc:492] T 780784f9ab7d4599a6a63b616d980c18 P ec9b702bb1e64be2aaa0fa97a8b649eb: Bootstrap starting.
I20260812 21:07:42.062397 13973 raft_consensus.cc:348] T 780784f9ab7d4599a6a63b616d980c18 P d0d15142d5fb46958bbd6808fcd22186 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7873e78bc42342c1994bba9de68b2578" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 36327 } } peers { permanent_uuid: "ec9b702bb1e64be2aaa0fa97a8b649eb" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 39145 } } peers { permanent_uuid: "d0d15142d5fb46958bbd6808fcd22186" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 37329 } }
I20260812 21:07:42.062536 13973 raft_consensus.cc:374] T 780784f9ab7d4599a6a63b616d980c18 P d0d15142d5fb46958bbd6808fcd22186 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:07:42.062577 13973 raft_consensus.cc:729] T 780784f9ab7d4599a6a63b616d980c18 P d0d15142d5fb46958bbd6808fcd22186 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0d15142d5fb46958bbd6808fcd22186, State: Initialized, Role: FOLLOWER
I20260812 21:07:42.062707 13973 consensus_queue.cc:260] T 780784f9ab7d4599a6a63b616d980c18 P d0d15142d5fb46958bbd6808fcd22186 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7873e78bc42342c1994bba9de68b2578" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 36327 } } peers { permanent_uuid: "ec9b702bb1e64be2aaa0fa97a8b649eb" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 39145 } } peers { permanent_uuid: "d0d15142d5fb46958bbd6808fcd22186" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 37329 } }
I20260812 21:07:42.062734 13972 tablet_bootstrap.cc:654] T 780784f9ab7d4599a6a63b616d980c18 P ec9b702bb1e64be2aaa0fa97a8b649eb: Neither blocks nor log segments found. Creating new log.
I20260812 21:07:42.062734 13975 tablet_bootstrap.cc:654] T 780784f9ab7d4599a6a63b616d980c18 P 7873e78bc42342c1994bba9de68b2578: Neither blocks nor log segments found. Creating new log.
I20260812 21:07:42.062948 13973 ts_tablet_manager.cc:1428] T 780784f9ab7d4599a6a63b616d980c18 P d0d15142d5fb46958bbd6808fcd22186: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 21:07:42.063062 13720 heartbeater.cc:503] Master 127.11.254.190:45081 was elected leader, sending a full tablet report...
I20260812 21:07:42.063045 13975 log.cc:824] T 780784f9ab7d4599a6a63b616d980c18 P 7873e78bc42342c1994bba9de68b2578: Log is configured to *not* fsync() on all Append() calls
I20260812 21:07:42.063047 13972 log.cc:824] T 780784f9ab7d4599a6a63b616d980c18 P ec9b702bb1e64be2aaa0fa97a8b649eb: Log is configured to *not* fsync() on all Append() calls
I20260812 21:07:42.063822 13972 tablet_bootstrap.cc:492] T 780784f9ab7d4599a6a63b616d980c18 P ec9b702bb1e64be2aaa0fa97a8b649eb: No bootstrap required, opened a new log
I20260812 21:07:42.063889 13972 ts_tablet_manager.cc:1397] T 780784f9ab7d4599a6a63b616d980c18 P ec9b702bb1e64be2aaa0fa97a8b649eb: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260812 21:07:42.064237 13975 tablet_bootstrap.cc:492] T 780784f9ab7d4599a6a63b616d980c18 P 7873e78bc42342c1994bba9de68b2578: No bootstrap required, opened a new log
I20260812 21:07:42.064299 13975 ts_tablet_manager.cc:1397] T 780784f9ab7d4599a6a63b616d980c18 P 7873e78bc42342c1994bba9de68b2578: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 21:07:42.065789 13972 raft_consensus.cc:348] T 780784f9ab7d4599a6a63b616d980c18 P ec9b702bb1e64be2aaa0fa97a8b649eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7873e78bc42342c1994bba9de68b2578" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 36327 } } peers { permanent_uuid: "ec9b702bb1e64be2aaa0fa97a8b649eb" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 39145 } } peers { permanent_uuid: "d0d15142d5fb46958bbd6808fcd22186" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 37329 } }
I20260812 21:07:42.065915 13972 raft_consensus.cc:374] T 780784f9ab7d4599a6a63b616d980c18 P ec9b702bb1e64be2aaa0fa97a8b649eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:07:42.065977 13972 raft_consensus.cc:729] T 780784f9ab7d4599a6a63b616d980c18 P ec9b702bb1e64be2aaa0fa97a8b649eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec9b702bb1e64be2aaa0fa97a8b649eb, State: Initialized, Role: FOLLOWER
I20260812 21:07:42.066103 13972 consensus_queue.cc:260] T 780784f9ab7d4599a6a63b616d980c18 P ec9b702bb1e64be2aaa0fa97a8b649eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7873e78bc42342c1994bba9de68b2578" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 36327 } } peers { permanent_uuid: "ec9b702bb1e64be2aaa0fa97a8b649eb" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 39145 } } peers { permanent_uuid: "d0d15142d5fb46958bbd6808fcd22186" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 37329 } }
I20260812 21:07:42.066197 13975 raft_consensus.cc:348] T 780784f9ab7d4599a6a63b616d980c18 P 7873e78bc42342c1994bba9de68b2578 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7873e78bc42342c1994bba9de68b2578" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 36327 } } peers { permanent_uuid: "ec9b702bb1e64be2aaa0fa97a8b649eb" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 39145 } } peers { permanent_uuid: "d0d15142d5fb46958bbd6808fcd22186" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 37329 } }
I20260812 21:07:42.066337 13972 ts_tablet_manager.cc:1428] T 780784f9ab7d4599a6a63b616d980c18 P ec9b702bb1e64be2aaa0fa97a8b649eb: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20260812 21:07:42.066337 13975 raft_consensus.cc:374] T 780784f9ab7d4599a6a63b616d980c18 P 7873e78bc42342c1994bba9de68b2578 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:07:42.066376 13975 raft_consensus.cc:729] T 780784f9ab7d4599a6a63b616d980c18 P 7873e78bc42342c1994bba9de68b2578 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7873e78bc42342c1994bba9de68b2578, State: Initialized, Role: FOLLOWER
I20260812 21:07:42.066408 13947 heartbeater.cc:503] Master 127.11.254.190:45081 was elected leader, sending a full tablet report...
I20260812 21:07:42.066480 13975 consensus_queue.cc:260] T 780784f9ab7d4599a6a63b616d980c18 P 7873e78bc42342c1994bba9de68b2578 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7873e78bc42342c1994bba9de68b2578" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 36327 } } peers { permanent_uuid: "ec9b702bb1e64be2aaa0fa97a8b649eb" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 39145 } } peers { permanent_uuid: "d0d15142d5fb46958bbd6808fcd22186" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 37329 } }
I20260812 21:07:42.066726 13975 ts_tablet_manager.cc:1428] T 780784f9ab7d4599a6a63b616d980c18 P 7873e78bc42342c1994bba9de68b2578: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260812 21:07:42.066767 13482 heartbeater.cc:503] Master 127.11.254.190:45081 was elected leader, sending a full tablet report...
I20260812 21:07:42.111865 13983 raft_consensus.cc:482] T 780784f9ab7d4599a6a63b616d980c18 P 7873e78bc42342c1994bba9de68b2578 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 21:07:42.112000 13983 raft_consensus.cc:504] T 780784f9ab7d4599a6a63b616d980c18 P 7873e78bc42342c1994bba9de68b2578 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7873e78bc42342c1994bba9de68b2578" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 36327 } } peers { permanent_uuid: "ec9b702bb1e64be2aaa0fa97a8b649eb" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 39145 } } peers { permanent_uuid: "d0d15142d5fb46958bbd6808fcd22186" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 37329 } }
I20260812 21:07:42.112367 13983 leader_election.cc:288] T 780784f9ab7d4599a6a63b616d980c18 P 7873e78bc42342c1994bba9de68b2578 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ec9b702bb1e64be2aaa0fa97a8b649eb (127.11.254.131:39145), d0d15142d5fb46958bbd6808fcd22186 (127.11.254.130:37329)
I20260812 21:07:42.115402 13971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:42.112631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.129:39427 (local address 127.11.254.131:39145)
0812 21:07:42.112696 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0812 21:07:42.112698 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:07:42.112707 (+ 9us) server_negotiation.cc:407] Connection header received
0812 21:07:42.112735 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:07:42.112736 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:07:42.112749 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:07:42.112798 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:07:42.113003 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:42.113605 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:42.114687 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:42.115064 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:42.115118 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:07:42.115124 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0812 21:07:42.115284 (+ 160us) server_negotiation.cc:299] Negotiation successful
0812 21:07:42.115325 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260812 21:07:42.115422 13484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:42.112550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.131:39145 (local address 127.11.254.129:39427)
0812 21:07:42.112609 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0812 21:07:42.112623 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 21:07:42.112679 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:07:42.112822 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:07:42.112823 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:07:42.112825 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:07:42.112902 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:07:42.112904 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:07:42.113680 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:07:42.113681 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:07:42.114594 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:07:42.114597 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:07:42.115145 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:07:42.115146 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:07:42.115216 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:07:42.115220 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 21:07:42.115263 (+ 43us) client_negotiation.cc:240] Negotiation successful
0812 21:07:42.115305 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260812 21:07:42.115417 13970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:42.112747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.254.129:55505 (local address 127.11.254.130:37329)
0812 21:07:42.112795 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0812 21:07:42.112797 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:07:42.112887 (+ 90us) server_negotiation.cc:407] Connection header received
0812 21:07:42.112937 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:07:42.112938 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:07:42.112948 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:07:42.113004 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 21:07:42.113158 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:42.113705 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:42.114735 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:42.115094 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:42.115163 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:07:42.115171 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 21:07:42.115309 (+ 138us) server_negotiation.cc:299] Negotiation successful
0812 21:07:42.115348 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 21:07:42.115435 13984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:42.112649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.130:37329 (local address 127.11.254.129:55505)
0812 21:07:42.112825 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0812 21:07:42.112834 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 21:07:42.112882 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:07:42.113013 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:07:42.113013 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:07:42.113015 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 21:07:42.113097 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:07:42.113098 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:07:42.113782 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:07:42.113783 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:07:42.114645 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:07:42.114648 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:07:42.115177 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:07:42.115178 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:07:42.115242 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:07:42.115246 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 21:07:42.115297 (+ 51us) client_negotiation.cc:240] Negotiation successful
0812 21:07:42.115333 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":107,"threads_started":1}
I20260812 21:07:42.115749 13869 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "780784f9ab7d4599a6a63b616d980c18" candidate_uuid: "7873e78bc42342c1994bba9de68b2578" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec9b702bb1e64be2aaa0fa97a8b649eb" is_pre_election: true
I20260812 21:07:42.115749 13627 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "780784f9ab7d4599a6a63b616d980c18" candidate_uuid: "7873e78bc42342c1994bba9de68b2578" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0d15142d5fb46958bbd6808fcd22186" is_pre_election: true
I20260812 21:07:42.115914 13627 raft_consensus.cc:2454] T 780784f9ab7d4599a6a63b616d980c18 P d0d15142d5fb46958bbd6808fcd22186 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7873e78bc42342c1994bba9de68b2578 in term 0.
I20260812 21:07:42.115914 13869 raft_consensus.cc:2454] T 780784f9ab7d4599a6a63b616d980c18 P ec9b702bb1e64be2aaa0fa97a8b649eb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7873e78bc42342c1994bba9de68b2578 in term 0.
I20260812 21:07:42.116147 13277 leader_election.cc:302] T 780784f9ab7d4599a6a63b616d980c18 P 7873e78bc42342c1994bba9de68b2578 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7873e78bc42342c1994bba9de68b2578, ec9b702bb1e64be2aaa0fa97a8b649eb; no voters:
I20260812 21:07:42.116276 13983 raft_consensus.cc:2781] T 780784f9ab7d4599a6a63b616d980c18 P 7873e78bc42342c1994bba9de68b2578 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 21:07:42.116335 13983 raft_consensus.cc:482] T 780784f9ab7d4599a6a63b616d980c18 P 7873e78bc42342c1994bba9de68b2578 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 21:07:42.116357 13983 raft_consensus.cc:3037] T 780784f9ab7d4599a6a63b616d980c18 P 7873e78bc42342c1994bba9de68b2578 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:07:42.117060 13983 raft_consensus.cc:504] T 780784f9ab7d4599a6a63b616d980c18 P 7873e78bc42342c1994bba9de68b2578 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7873e78bc42342c1994bba9de68b2578" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 36327 } } peers { permanent_uuid: "ec9b702bb1e64be2aaa0fa97a8b649eb" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 39145 } } peers { permanent_uuid: "d0d15142d5fb46958bbd6808fcd22186" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 37329 } }
I20260812 21:07:42.117177 13983 leader_election.cc:288] T 780784f9ab7d4599a6a63b616d980c18 P 7873e78bc42342c1994bba9de68b2578 [CANDIDATE]: Term 1 election: Requested vote from peers ec9b702bb1e64be2aaa0fa97a8b649eb (127.11.254.131:39145), d0d15142d5fb46958bbd6808fcd22186 (127.11.254.130:37329)
I20260812 21:07:42.117370 13869 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "780784f9ab7d4599a6a63b616d980c18" candidate_uuid: "7873e78bc42342c1994bba9de68b2578" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec9b702bb1e64be2aaa0fa97a8b649eb"
I20260812 21:07:42.117414 13627 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "780784f9ab7d4599a6a63b616d980c18" candidate_uuid: "7873e78bc42342c1994bba9de68b2578" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0d15142d5fb46958bbd6808fcd22186"
I20260812 21:07:42.117453 13869 raft_consensus.cc:3037] T 780784f9ab7d4599a6a63b616d980c18 P ec9b702bb1e64be2aaa0fa97a8b649eb [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:07:42.117501 13627 raft_consensus.cc:3037] T 780784f9ab7d4599a6a63b616d980c18 P d0d15142d5fb46958bbd6808fcd22186 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:07:42.118269 13627 raft_consensus.cc:2454] T 780784f9ab7d4599a6a63b616d980c18 P d0d15142d5fb46958bbd6808fcd22186 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7873e78bc42342c1994bba9de68b2578 in term 1.
I20260812 21:07:42.118270 13869 raft_consensus.cc:2454] T 780784f9ab7d4599a6a63b616d980c18 P ec9b702bb1e64be2aaa0fa97a8b649eb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7873e78bc42342c1994bba9de68b2578 in term 1.
I20260812 21:07:42.118446 13277 leader_election.cc:302] T 780784f9ab7d4599a6a63b616d980c18 P 7873e78bc42342c1994bba9de68b2578 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7873e78bc42342c1994bba9de68b2578, d0d15142d5fb46958bbd6808fcd22186; no voters:
I20260812 21:07:42.118549 13983 raft_consensus.cc:2781] T 780784f9ab7d4599a6a63b616d980c18 P 7873e78bc42342c1994bba9de68b2578 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:07:42.118706 13983 raft_consensus.cc:686] T 780784f9ab7d4599a6a63b616d980c18 P 7873e78bc42342c1994bba9de68b2578 [term 1 LEADER]: Becoming Leader. State: Replica: 7873e78bc42342c1994bba9de68b2578, State: Running, Role: LEADER
I20260812 21:07:42.118798 13983 consensus_queue.cc:237] T 780784f9ab7d4599a6a63b616d980c18 P 7873e78bc42342c1994bba9de68b2578 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7873e78bc42342c1994bba9de68b2578" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 36327 } } peers { permanent_uuid: "ec9b702bb1e64be2aaa0fa97a8b649eb" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 39145 } } peers { permanent_uuid: "d0d15142d5fb46958bbd6808fcd22186" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 37329 } }
I20260812 21:07:42.119746 13152 catalog_manager.cc:5495] T 780784f9ab7d4599a6a63b616d980c18 P 7873e78bc42342c1994bba9de68b2578 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7873e78bc42342c1994bba9de68b2578 (127.11.254.129). New cstate: current_term: 1 leader_uuid: "7873e78bc42342c1994bba9de68b2578" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7873e78bc42342c1994bba9de68b2578" member_type: VOTER last_known_addr { host: "127.11.254.129" port: 36327 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec9b702bb1e64be2aaa0fa97a8b649eb" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 39145 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d0d15142d5fb46958bbd6808fcd22186" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 37329 } health_report { overall_health: UNKNOWN } } }
I20260812 21:07:42.130154 13963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:42.127867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58844 (local address 127.11.254.129:36327)
0812 21:07:42.127930 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0812 21:07:42.127932 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:07:42.127940 (+ 8us) server_negotiation.cc:407] Connection header received
0812 21:07:42.127978 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:07:42.127978 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:07:42.127997 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:07:42.128062 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 21:07:42.128168 (+ 106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:42.128640 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:42.129386 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:42.129582 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:42.129638 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:07:42.129787 (+ 149us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 21:07:42.129894 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 21:07:42.129989 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0812 21:07:42.130024 (+ 35us) server_negotiation.cc:299] Negotiation successful
0812 21:07:42.130053 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260812 21:07:42.131964 13627 raft_consensus.cc:1261] T 780784f9ab7d4599a6a63b616d980c18 P d0d15142d5fb46958bbd6808fcd22186 [term 1 FOLLOWER]: Refusing update from remote peer 7873e78bc42342c1994bba9de68b2578: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 21:07:42.131989 13869 raft_consensus.cc:1261] T 780784f9ab7d4599a6a63b616d980c18 P ec9b702bb1e64be2aaa0fa97a8b649eb [term 1 FOLLOWER]: Refusing update from remote peer 7873e78bc42342c1994bba9de68b2578: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 21:07:42.132313 13983 consensus_queue.cc:1035] T 780784f9ab7d4599a6a63b616d980c18 P 7873e78bc42342c1994bba9de68b2578 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec9b702bb1e64be2aaa0fa97a8b649eb" member_type: VOTER last_known_addr { host: "127.11.254.131" port: 39145 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 21:07:42.132469 13985 consensus_queue.cc:1035] T 780784f9ab7d4599a6a63b616d980c18 P 7873e78bc42342c1994bba9de68b2578 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0d15142d5fb46958bbd6808fcd22186" member_type: VOTER last_known_addr { host: "127.11.254.130" port: 37329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 21:07:42.134943 13989 mvcc.cc:205] Tried to move back new op lower bound from 7317786059289780224 to 7317786059239448576. Current Snapshot: MvccSnapshot[applied={T|T < 7317786059289780224}]
I20260812 21:07:42.136045 13990 mvcc.cc:205] Tried to move back new op lower bound from 7317786059289780224 to 7317786059239448576. Current Snapshot: MvccSnapshot[applied={T|T < 7317786059289780224 or (T in {7317786059289780224})}]
I20260812 21:07:42.137418 13154 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34014:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 21:07:42.137578 13154 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34014:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 21:07:42.138365 13154 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c53674e2465e403eaed972fb7e348d61: Sending DeleteTablet for 3 replicas of tablet 780784f9ab7d4599a6a63b616d980c18
I20260812 21:07:42.138690 13359 tablet_service.cc:1514] Processing DeleteTablet for tablet 780784f9ab7d4599a6a63b616d980c18 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:07:42 UTC) from {username='slave'} at 127.0.0.1:58834
I20260812 21:07:42.138723 13831 tablet_service.cc:1514] Processing DeleteTablet for tablet 780784f9ab7d4599a6a63b616d980c18 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:07:42 UTC) from {username='slave'} at 127.0.0.1:49638
I20260812 21:07:42.138918 13999 tablet_replica.cc:330] T 780784f9ab7d4599a6a63b616d980c18 P 7873e78bc42342c1994bba9de68b2578: stopping tablet replica
I20260812 21:07:42.138969 13590 tablet_service.cc:1514] Processing DeleteTablet for tablet 780784f9ab7d4599a6a63b616d980c18 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 21:07:42 UTC) from {username='slave'} at 127.0.0.1:40286
I20260812 21:07:42.139014 13999 raft_consensus.cc:2229] T 780784f9ab7d4599a6a63b616d980c18 P 7873e78bc42342c1994bba9de68b2578 [term 1 LEADER]: Raft consensus shutting down.
I20260812 21:07:42.139103 13999 raft_consensus.cc:2258] T 780784f9ab7d4599a6a63b616d980c18 P 7873e78bc42342c1994bba9de68b2578 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 21:07:42.139295 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 13254
I20260812 21:07:42.139328 14007 tablet_replica.cc:330] T 780784f9ab7d4599a6a63b616d980c18 P d0d15142d5fb46958bbd6808fcd22186: stopping tablet replica
I20260812 21:07:42.144203 14007 raft_consensus.cc:2229] T 780784f9ab7d4599a6a63b616d980c18 P d0d15142d5fb46958bbd6808fcd22186 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 21:07:42.144387 14007 raft_consensus.cc:2258] T 780784f9ab7d4599a6a63b616d980c18 P d0d15142d5fb46958bbd6808fcd22186 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 21:07:42.147241 14007 ts_tablet_manager.cc:1905] T 780784f9ab7d4599a6a63b616d980c18 P d0d15142d5fb46958bbd6808fcd22186: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 21:07:42.149796 14007 ts_tablet_manager.cc:1918] T 780784f9ab7d4599a6a63b616d980c18 P d0d15142d5fb46958bbd6808fcd22186: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 21:07:42.149900 14007 log.cc:1196] T 780784f9ab7d4599a6a63b616d980c18 P d0d15142d5fb46958bbd6808fcd22186: Deleting WAL directory at /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786568607354921-12282-0/minicluster-data/ts-1/wal/wals/780784f9ab7d4599a6a63b616d980c18
I20260812 21:07:42.150240 14007 ts_tablet_manager.cc:1939] T 780784f9ab7d4599a6a63b616d980c18 P d0d15142d5fb46958bbd6808fcd22186: Deleting consensus metadata
I20260812 21:07:42.152072 14005 tablet_replica.cc:330] T 780784f9ab7d4599a6a63b616d980c18 P ec9b702bb1e64be2aaa0fa97a8b649eb: stopping tablet replica
I20260812 21:07:42.152209 14005 raft_consensus.cc:2229] T 780784f9ab7d4599a6a63b616d980c18 P ec9b702bb1e64be2aaa0fa97a8b649eb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 21:07:42.152320 14005 raft_consensus.cc:2258] T 780784f9ab7d4599a6a63b616d980c18 P ec9b702bb1e64be2aaa0fa97a8b649eb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 21:07:42.154646 14005 ts_tablet_manager.cc:1905] T 780784f9ab7d4599a6a63b616d980c18 P ec9b702bb1e64be2aaa0fa97a8b649eb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 21:07:42.156461 14005 ts_tablet_manager.cc:1918] T 780784f9ab7d4599a6a63b616d980c18 P ec9b702bb1e64be2aaa0fa97a8b649eb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 21:07:42.156558 14005 log.cc:1196] T 780784f9ab7d4599a6a63b616d980c18 P ec9b702bb1e64be2aaa0fa97a8b649eb: Deleting WAL directory at /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786568607354921-12282-0/minicluster-data/ts-2/wal/wals/780784f9ab7d4599a6a63b616d980c18
I20260812 21:07:42.156865 14005 ts_tablet_manager.cc:1939] T 780784f9ab7d4599a6a63b616d980c18 P ec9b702bb1e64be2aaa0fa97a8b649eb: Deleting consensus metadata
I20260812 21:07:42.157109 13131 catalog_manager.cc:4841] TS d0d15142d5fb46958bbd6808fcd22186 (127.11.254.130:37329): tablet 780784f9ab7d4599a6a63b616d980c18 (table test-table [id=46062005ed2e42f5a5a4c88c4a6841c6]) successfully deleted
I20260812 21:07:42.157405 13131 catalog_manager.cc:4841] TS ec9b702bb1e64be2aaa0fa97a8b649eb (127.11.254.131:39145): tablet 780784f9ab7d4599a6a63b616d980c18 (table test-table [id=46062005ed2e42f5a5a4c88c4a6841c6]) successfully deleted
W20260812 21:07:42.161368 13131 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.254.129:36327 (error 108)
I20260812 21:07:42.161656 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 13486
I20260812 21:07:42.162010 13960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:42.161838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.129:36327 (local address 127.0.0.1:58854)
0812 21:07:42.161902 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0812 21:07:42.161943 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.129:36327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260812 21:07:42.172317 13131 catalog_manager.cc:4584] TS 7873e78bc42342c1994bba9de68b2578 (127.11.254.129:36327): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 780784f9ab7d4599a6a63b616d980c18: Network error: Client connection negotiation failed: client connection to 127.11.254.129:36327: connect: Connection refused (error 111)
I20260812 21:07:42.180951 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 13727
I20260812 21:07:42.192672 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 13106
2026-08-12T21:07:42Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (769 ms)
[----------] 6 tests from EncryptionPolicyTest (9082 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786568607354921-12282-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 21:07:42 dist-test-slave-vpvm krb5kdc[14011](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 21:07:42 dist-test-slave-vpvm krb5kdc[14011](info): set up 2 sockets
Aug 12 21:07:42 dist-test-slave-vpvm krb5kdc[14011](info): commencing operation
krb5kdc: starting...
W20260812 21:07:43.387835 12282 mini_kdc.cc:121] Time spent starting KDC: real 1.167s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786568607354921-12282-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786568607354921-12282-0/krb5kdc/test-user.keytab.
Aug 12 21:07:43 dist-test-slave-vpvm krb5kdc[14011](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568863, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T21:07:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T21:07:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786568607354921-12282-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786568607354921-12282-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 21:07:43.514797 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786568607354921-12282-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786568607354921-12282-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786568607354921-12282-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41995
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786568607354921-12282-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36741
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41995
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786568607354921-12282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786568607354921-12282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786568607354921-12282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:07:43.597781 14029 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:07:43.598059 14029 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:07:43.598106 14029 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:07:43.599318 14029 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 21:07:43.599363 14029 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:07:43.599383 14029 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:07:43.599399 14029 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 21:07:43.599416 14029 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 21:07:43.600953 14029 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36741
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786568607354921-12282-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786568607354921-12282-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41995
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786568607354921-12282-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41995
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14029
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786568607354921-12282-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:07:43.601229 14029 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:07:43.601455 14029 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 21:07:43.606487 14029 server_base.cc:1034] running on GCE node
W20260812 21:07:43.606410 14039 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:07:43.606395 14042 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:07:43.606381 14038 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:07:43.607072 14029 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:07:43.607370 14029 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:07:43.608565 14029 hybrid_clock.cc:648] HybridClock initialized: now 1786568863608549 us; error 30 us; skew 500 ppm
Aug 12 21:07:43 dist-test-slave-vpvm krb5kdc[14011](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568863, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:07:43.615846 14029 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 21:07:43.616370 14029 webserver.cc:460] Webserver started at http://127.0.0.1:39723/ using document root <none> and password file <none>
I20260812 21:07:43.616621 14029 fs_manager.cc:359] Metadata directory not provided
I20260812 21:07:43.616674 14029 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:07:43.616803 14029 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:07:43.618003 14029 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786568607354921-12282-0/minicluster-data/master-0/data/instance:
uuid: "fe207c35802a44dbb437f718bde8b62d"
format_stamp: "Formatted at 2026-08-12 21:07:43 on dist-test-slave-vpvm"
server_key: "bd7ad0cbbae4f13bdd7a8441280420c4"
server_key_iv: "7f2c2581458d4016f98b0c33ff044073"
server_key_version: "encryptionkey@0"
I20260812 21:07:43.618392 14029 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786568607354921-12282-0/minicluster-data/master-0/wal/instance:
uuid: "fe207c35802a44dbb437f718bde8b62d"
format_stamp: "Formatted at 2026-08-12 21:07:43 on dist-test-slave-vpvm"
server_key: "bd7ad0cbbae4f13bdd7a8441280420c4"
server_key_iv: "7f2c2581458d4016f98b0c33ff044073"
server_key_version: "encryptionkey@0"
I20260812 21:07:43.619834 14029 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 21:07:43.620684 14057 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:07:43.620903 14029 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 21:07:43.621004 14029 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786568607354921-12282-0/minicluster-data/master-0/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786568607354921-12282-0/minicluster-data/master-0/wal
uuid: "fe207c35802a44dbb437f718bde8b62d"
format_stamp: "Formatted at 2026-08-12 21:07:43 on dist-test-slave-vpvm"
server_key: "bd7ad0cbbae4f13bdd7a8441280420c4"
server_key_iv: "7f2c2581458d4016f98b0c33ff044073"
server_key_version: "encryptionkey@0"
I20260812 21:07:43.621079 14029 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786568607354921-12282-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786568607354921-12282-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786568607354921-12282-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:07:43.631793 14029 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:07:43.633809 14029 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:07:43.634016 14029 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:07:43.637914 14029 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41995
I20260812 21:07:43.637976 14167 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41995 every 8 connection(s)
I20260812 21:07:43.638331 14029 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
I20260812 21:07:43.639220 14169 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:07:43.640733 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 14029
I20260812 21:07:43.640833 12282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786568607354921-12282-0/minicluster-data/master-0/wal/instance
I20260812 21:07:43.641038 12282 external_mini_cluster.cc:1428] Setting key 9750fae190cedb11f750ae6b022e0aee
I20260812 21:07:43.641667 14169 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fe207c35802a44dbb437f718bde8b62d: Bootstrap starting.
I20260812 21:07:43.642339 14169 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fe207c35802a44dbb437f718bde8b62d: Neither blocks nor log segments found. Creating new log.
I20260812 21:07:43.642611 14169 log.cc:824] T 00000000000000000000000000000000 P fe207c35802a44dbb437f718bde8b62d: Log is configured to *not* fsync() on all Append() calls
I20260812 21:07:43.643292 14169 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fe207c35802a44dbb437f718bde8b62d: No bootstrap required, opened a new log
I20260812 21:07:43.645083 14169 raft_consensus.cc:348] T 00000000000000000000000000000000 P fe207c35802a44dbb437f718bde8b62d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe207c35802a44dbb437f718bde8b62d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41995 } }
I20260812 21:07:43.645226 14169 raft_consensus.cc:374] T 00000000000000000000000000000000 P fe207c35802a44dbb437f718bde8b62d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:07:43.645257 14169 raft_consensus.cc:729] T 00000000000000000000000000000000 P fe207c35802a44dbb437f718bde8b62d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fe207c35802a44dbb437f718bde8b62d, State: Initialized, Role: FOLLOWER
I20260812 21:07:43.645385 14169 consensus_queue.cc:260] T 00000000000000000000000000000000 P fe207c35802a44dbb437f718bde8b62d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe207c35802a44dbb437f718bde8b62d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41995 } }
I20260812 21:07:43.645465 14169 raft_consensus.cc:388] T 00000000000000000000000000000000 P fe207c35802a44dbb437f718bde8b62d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 21:07:43.645496 14169 raft_consensus.cc:482] T 00000000000000000000000000000000 P fe207c35802a44dbb437f718bde8b62d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 21:07:43.645563 14169 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fe207c35802a44dbb437f718bde8b62d [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:07:43.646350 14169 raft_consensus.cc:504] T 00000000000000000000000000000000 P fe207c35802a44dbb437f718bde8b62d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe207c35802a44dbb437f718bde8b62d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41995 } }
I20260812 21:07:43.646471 14169 leader_election.cc:302] T 00000000000000000000000000000000 P fe207c35802a44dbb437f718bde8b62d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fe207c35802a44dbb437f718bde8b62d; no voters:
I20260812 21:07:43.646665 14169 leader_election.cc:288] T 00000000000000000000000000000000 P fe207c35802a44dbb437f718bde8b62d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 21:07:43.646728 14178 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fe207c35802a44dbb437f718bde8b62d [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:07:43.646937 14169 sys_catalog.cc:564] T 00000000000000000000000000000000 P fe207c35802a44dbb437f718bde8b62d [sys.catalog]: configured and running, proceeding with master startup.
I20260812 21:07:43.646986 14178 raft_consensus.cc:686] T 00000000000000000000000000000000 P fe207c35802a44dbb437f718bde8b62d [term 1 LEADER]: Becoming Leader. State: Replica: fe207c35802a44dbb437f718bde8b62d, State: Running, Role: LEADER
I20260812 21:07:43.647168 14178 consensus_queue.cc:237] T 00000000000000000000000000000000 P fe207c35802a44dbb437f718bde8b62d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe207c35802a44dbb437f718bde8b62d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41995 } }
I20260812 21:07:43.647594 14180 sys_catalog.cc:455] T 00000000000000000000000000000000 P fe207c35802a44dbb437f718bde8b62d [sys.catalog]: SysCatalogTable state changed. Reason: New leader fe207c35802a44dbb437f718bde8b62d. Latest consensus state: current_term: 1 leader_uuid: "fe207c35802a44dbb437f718bde8b62d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe207c35802a44dbb437f718bde8b62d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41995 } } }
I20260812 21:07:43.647614 14179 sys_catalog.cc:455] T 00000000000000000000000000000000 P fe207c35802a44dbb437f718bde8b62d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fe207c35802a44dbb437f718bde8b62d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe207c35802a44dbb437f718bde8b62d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41995 } } }
I20260812 21:07:43.647771 14180 sys_catalog.cc:458] T 00000000000000000000000000000000 P fe207c35802a44dbb437f718bde8b62d [sys.catalog]: This master's current role is: LEADER
I20260812 21:07:43.647799 14179 sys_catalog.cc:458] T 00000000000000000000000000000000 P fe207c35802a44dbb437f718bde8b62d [sys.catalog]: This master's current role is: LEADER
Aug 12 21:07:43 dist-test-slave-vpvm krb5kdc[14011](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568863, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260812 21:07:43.648025 14193 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 21:07:43.648525 14193 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 21:07:43.650543 14193 catalog_manager.cc:1347] Generated new cluster ID: 9dbfb0b90c224a78ae5219da2b418da0
I20260812 21:07:43.650599 14193 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 21:07:43.652521 14172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:43.641601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37329 (local address 127.0.0.1:41995)
0812 21:07:43.641825 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0812 21:07:43.641836 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 21:07:43.641872 (+ 36us) server_negotiation.cc:407] Connection header received
0812 21:07:43.642360 (+ 488us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:07:43.642367 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:07:43.642431 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:07:43.642628 (+ 197us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:07:43.643301 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:43.644257 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:43.644990 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:43.645481 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:43.645601 (+ 120us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:07:43.648585 (+ 2984us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:07:43.648589 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:07:43.648590 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:07:43.648603 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:07:43.650783 (+ 2180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:07:43.651237 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:07:43.651240 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:07:43.651240 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:07:43.651356 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:07:43.651575 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:07:43.651576 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:07:43.651576 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:07:43.651915 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:07:43.652088 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0812 21:07:43.652175 (+ 87us) server_negotiation.cc:299] Negotiation successful
0812 21:07:43.652285 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":95,"threads_started":1}
I20260812 21:07:43.653580 12282 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 21:07:43.665330 14172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:43.655220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52619 (local address 127.0.0.1:41995)
0812 21:07:43.655257 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0812 21:07:43.655258 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 21:07:43.658122 (+ 2864us) server_negotiation.cc:407] Connection header received
0812 21:07:43.658181 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:07:43.658182 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:07:43.658195 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:07:43.658248 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:07:43.658878 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:43.659677 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:43.660660 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:43.661026 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:43.661091 (+ 65us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:07:43.662526 (+ 1435us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:07:43.662527 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:07:43.662528 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:07:43.662536 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:07:43.664103 (+ 1567us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:07:43.664504 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:07:43.664505 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:07:43.664505 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:07:43.664568 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:07:43.664772 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:07:43.664772 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:07:43.664773 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:07:43.664922 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:07:43.665028 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0812 21:07:43.665131 (+ 103us) server_negotiation.cc:299] Negotiation successful
0812 21:07:43.665224 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260812 21:07:43.682240 14193 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 21:07:43.682797 14193 catalog_manager.cc:1504] Loading token signing keys...
I20260812 21:07:43.699363 14193 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fe207c35802a44dbb437f718bde8b62d: Generated new TSK 0
I20260812 21:07:43.699676 14193 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 21:07:43.713238 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 14029
W20260812 21:07:43.727140 14009 connection.cc:535] client connection to 127.0.0.1:41995 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41995): Connection reset by peer (error 104)
2026-08-12T21:07:43Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1520 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786568607354921-12282-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 21:07:43 dist-test-slave-vpvm krb5kdc[14220](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 21:07:43 dist-test-slave-vpvm krb5kdc[14220](info): set up 2 sockets
Aug 12 21:07:43 dist-test-slave-vpvm krb5kdc[14220](info): commencing operation
krb5kdc: starting...
W20260812 21:07:44.894300 12282 mini_kdc.cc:121] Time spent starting KDC: real 1.154s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786568607354921-12282-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786568607354921-12282-0/krb5kdc/test-user.keytab.
Aug 12 21:07:44 dist-test-slave-vpvm krb5kdc[14220](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568864, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T21:07:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T21:07:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786568607354921-12282-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786568607354921-12282-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 21:07:45.018998 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786568607354921-12282-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786568607354921-12282-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786568607354921-12282-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43861
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786568607354921-12282-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38643
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43861
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786568607354921-12282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786568607354921-12282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786568607354921-12282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:07:45.101522 14245 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:07:45.101747 14245 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:07:45.101779 14245 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:07:45.102950 14245 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 21:07:45.102993 14245 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:07:45.103011 14245 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:07:45.103027 14245 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 21:07:45.103042 14245 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 21:07:45.104547 14245 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38643
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786568607354921-12282-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786568607354921-12282-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43861
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786568607354921-12282-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43861
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14245
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786568607354921-12282-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:07:45.104813 14245 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:07:45.105053 14245 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:07:45.109891 14255 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:07:45.109902 14245 server_base.cc:1034] running on GCE node
W20260812 21:07:45.109867 14253 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:07:45.109867 14259 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:07:45.110445 14245 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:07:45.110716 14245 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:07:45.111862 14245 hybrid_clock.cc:648] HybridClock initialized: now 1786568865111846 us; error 29 us; skew 500 ppm
Aug 12 21:07:45 dist-test-slave-vpvm krb5kdc[14220](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568865, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:07:45.114357 14245 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 21:07:45.114799 14245 webserver.cc:460] Webserver started at http://127.0.0.1:34465/ using document root <none> and password file <none>
I20260812 21:07:45.115018 14245 fs_manager.cc:359] Metadata directory not provided
I20260812 21:07:45.115068 14245 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:07:45.115195 14245 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:07:45.116259 14245 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786568607354921-12282-0/minicluster-data/master-0/data/instance:
uuid: "6e6abc4b11f54d9590da56fb8fa2567d"
format_stamp: "Formatted at 2026-08-12 21:07:45 on dist-test-slave-vpvm"
server_key: "98c966b5f27828ead1a84f49fb698d96"
server_key_iv: "a2a99dceefa74777f6ae630ab153d486"
server_key_version: "encryptionkey@0"
I20260812 21:07:45.116600 14245 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786568607354921-12282-0/minicluster-data/master-0/wal/instance:
uuid: "6e6abc4b11f54d9590da56fb8fa2567d"
format_stamp: "Formatted at 2026-08-12 21:07:45 on dist-test-slave-vpvm"
server_key: "98c966b5f27828ead1a84f49fb698d96"
server_key_iv: "a2a99dceefa74777f6ae630ab153d486"
server_key_version: "encryptionkey@0"
I20260812 21:07:45.117913 14245 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260812 21:07:45.118698 14274 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:07:45.118872 14245 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 21:07:45.118958 14245 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786568607354921-12282-0/minicluster-data/master-0/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786568607354921-12282-0/minicluster-data/master-0/wal
uuid: "6e6abc4b11f54d9590da56fb8fa2567d"
format_stamp: "Formatted at 2026-08-12 21:07:45 on dist-test-slave-vpvm"
server_key: "98c966b5f27828ead1a84f49fb698d96"
server_key_iv: "a2a99dceefa74777f6ae630ab153d486"
server_key_version: "encryptionkey@0"
I20260812 21:07:45.119024 14245 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786568607354921-12282-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786568607354921-12282-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786568607354921-12282-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:07:45.138190 14245 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:07:45.140407 14245 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:07:45.140566 14245 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:07:45.144600 14245 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43861
I20260812 21:07:45.144659 14362 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43861 every 8 connection(s)
I20260812 21:07:45.144987 14245 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
I20260812 21:07:45.145843 14363 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:07:45.148162 14363 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e6abc4b11f54d9590da56fb8fa2567d: Bootstrap starting.
I20260812 21:07:45.148708 14363 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6e6abc4b11f54d9590da56fb8fa2567d: Neither blocks nor log segments found. Creating new log.
I20260812 21:07:45.148945 14363 log.cc:824] T 00000000000000000000000000000000 P 6e6abc4b11f54d9590da56fb8fa2567d: Log is configured to *not* fsync() on all Append() calls
I20260812 21:07:45.149595 14363 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e6abc4b11f54d9590da56fb8fa2567d: No bootstrap required, opened a new log
I20260812 21:07:45.151224 14363 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6e6abc4b11f54d9590da56fb8fa2567d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e6abc4b11f54d9590da56fb8fa2567d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43861 } }
I20260812 21:07:45.151371 14363 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6e6abc4b11f54d9590da56fb8fa2567d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:07:45.151392 14363 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6e6abc4b11f54d9590da56fb8fa2567d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e6abc4b11f54d9590da56fb8fa2567d, State: Initialized, Role: FOLLOWER
I20260812 21:07:45.151505 14363 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6e6abc4b11f54d9590da56fb8fa2567d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e6abc4b11f54d9590da56fb8fa2567d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43861 } }
I20260812 21:07:45.151582 14363 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6e6abc4b11f54d9590da56fb8fa2567d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 21:07:45.151613 14363 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6e6abc4b11f54d9590da56fb8fa2567d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 21:07:45.151651 14363 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6e6abc4b11f54d9590da56fb8fa2567d [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:07:45.152138 14363 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6e6abc4b11f54d9590da56fb8fa2567d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e6abc4b11f54d9590da56fb8fa2567d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43861 } }
I20260812 21:07:45.152225 14363 leader_election.cc:302] T 00000000000000000000000000000000 P 6e6abc4b11f54d9590da56fb8fa2567d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6e6abc4b11f54d9590da56fb8fa2567d; no voters:
I20260812 21:07:45.152348 14363 leader_election.cc:288] T 00000000000000000000000000000000 P 6e6abc4b11f54d9590da56fb8fa2567d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 21:07:45.152395 14368 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6e6abc4b11f54d9590da56fb8fa2567d [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:07:45.152575 14363 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6e6abc4b11f54d9590da56fb8fa2567d [sys.catalog]: configured and running, proceeding with master startup.
I20260812 21:07:45.152580 14368 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6e6abc4b11f54d9590da56fb8fa2567d [term 1 LEADER]: Becoming Leader. State: Replica: 6e6abc4b11f54d9590da56fb8fa2567d, State: Running, Role: LEADER
I20260812 21:07:45.152745 14368 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6e6abc4b11f54d9590da56fb8fa2567d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e6abc4b11f54d9590da56fb8fa2567d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43861 } }
I20260812 21:07:45.153153 14369 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e6abc4b11f54d9590da56fb8fa2567d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6e6abc4b11f54d9590da56fb8fa2567d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e6abc4b11f54d9590da56fb8fa2567d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43861 } } }
I20260812 21:07:45.153259 14370 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e6abc4b11f54d9590da56fb8fa2567d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6e6abc4b11f54d9590da56fb8fa2567d. Latest consensus state: current_term: 1 leader_uuid: "6e6abc4b11f54d9590da56fb8fa2567d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e6abc4b11f54d9590da56fb8fa2567d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43861 } } }
I20260812 21:07:45.153304 14369 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e6abc4b11f54d9590da56fb8fa2567d [sys.catalog]: This master's current role is: LEADER
I20260812 21:07:45.153795 14391 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 21:07:45.154016 14370 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e6abc4b11f54d9590da56fb8fa2567d [sys.catalog]: This master's current role is: LEADER
I20260812 21:07:45.154321 14391 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 21:07:45.154794 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 14245
I20260812 21:07:45.154917 12282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786568607354921-12282-0/minicluster-data/master-0/wal/instance
I20260812 21:07:45.155109 12282 external_mini_cluster.cc:1428] Setting key b2e34c9fd85202c0fb826563d143a7bc
I20260812 21:07:45.156220 14391 catalog_manager.cc:1347] Generated new cluster ID: c9a528b1c74046fab25de7222503cbd2
I20260812 21:07:45.156273 14391 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 21:07:45.167351 14414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:45.155541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47509 (local address 127.0.0.1:43861)
0812 21:07:45.155841 (+ 300us) server_negotiation.cc:206] Beginning negotiation
0812 21:07:45.155849 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 21:07:45.155875 (+ 26us) server_negotiation.cc:407] Connection header received
0812 21:07:45.155994 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:07:45.156002 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:07:45.156067 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:07:45.162156 (+ 6089us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:07:45.162409 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:45.163859 (+ 1450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:45.165014 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:45.165685 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:45.165796 (+ 111us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:07:45.166585 (+ 789us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:07:45.166590 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:07:45.166590 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 21:07:45.166821 (+ 231us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:07:45.166863 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 21:07:45.166878 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:07:45.166960 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0812 21:07:45.167046 (+ 86us) server_negotiation.cc:299] Negotiation successful
0812 21:07:45.167080 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":255,"thread_start_us":65,"threads_started":1}
I20260812 21:07:45.168341 12282 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 21:07:45.173677 14414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:45.170024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41885 (local address 127.0.0.1:43861)
0812 21:07:45.170065 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0812 21:07:45.170067 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:07:45.170124 (+ 57us) server_negotiation.cc:407] Connection header received
0812 21:07:45.170195 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:07:45.170196 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:07:45.170207 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:07:45.170267 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:07:45.170391 (+ 124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:45.171131 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:45.172151 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:45.172467 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:45.172527 (+ 60us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:07:45.173391 (+ 864us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:07:45.173393 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:07:45.173393 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 21:07:45.173398 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:07:45.173417 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 21:07:45.173422 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:07:45.173477 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0812 21:07:45.173577 (+ 100us) server_negotiation.cc:299] Negotiation successful
0812 21:07:45.173591 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 21:07:45.177733 14391 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 21:07:45.178306 14391 catalog_manager.cc:1504] Loading token signing keys...
I20260812 21:07:45.183461 14391 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6e6abc4b11f54d9590da56fb8fa2567d: Generated new TSK 0
I20260812 21:07:45.183658 14391 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 21:07:45.191414 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 14245
W20260812 21:07:45.202759 14216 connection.cc:535] client connection to 127.0.0.1:43861 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43861): Connection reset by peer (error 104)
2026-08-12T21:07:45Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1475 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786568607354921-12282-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 21:07:45 dist-test-slave-vpvm krb5kdc[14429](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 21:07:45 dist-test-slave-vpvm krb5kdc[14429](info): set up 2 sockets
Aug 12 21:07:45 dist-test-slave-vpvm krb5kdc[14429](info): commencing operation
krb5kdc: starting...
W20260812 21:07:46.377620 12282 mini_kdc.cc:121] Time spent starting KDC: real 1.161s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786568607354921-12282-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786568607354921-12282-0/krb5kdc/test-user.keytab.
Aug 12 21:07:46 dist-test-slave-vpvm krb5kdc[14429](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568866, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T21:07:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T21:07:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786568607354921-12282-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786568607354921-12282-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 21:07:46.505731 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786568607354921-12282-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786568607354921-12282-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786568607354921-12282-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35743
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786568607354921-12282-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41153
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35743
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786568607354921-12282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786568607354921-12282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786568607354921-12282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:07:46.587203 14450 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:07:46.587415 14450 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:07:46.587445 14450 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:07:46.588577 14450 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 21:07:46.588616 14450 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:07:46.588634 14450 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:07:46.588649 14450 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 21:07:46.588665 14450 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 21:07:46.590147 14450 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41153
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786568607354921-12282-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786568607354921-12282-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35743
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786568607354921-12282-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35743
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14450
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786568607354921-12282-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:07:46.590394 14450 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:07:46.590615 14450 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:07:46.595526 14459 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:07:46.595578 14467 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:07:46.595578 14462 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:07:46.595808 14450 server_base.cc:1034] running on GCE node
I20260812 21:07:46.596052 14450 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:07:46.596331 14450 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:07:46.597470 14450 hybrid_clock.cc:648] HybridClock initialized: now 1786568866597455 us; error 39 us; skew 500 ppm
Aug 12 21:07:46 dist-test-slave-vpvm krb5kdc[14429](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568866, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:07:46.600258 14450 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 21:07:46.600771 14450 webserver.cc:460] Webserver started at http://127.0.0.1:38567/ using document root <none> and password file <none>
I20260812 21:07:46.600977 14450 fs_manager.cc:359] Metadata directory not provided
I20260812 21:07:46.601020 14450 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:07:46.601141 14450 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:07:46.602313 14450 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786568607354921-12282-0/minicluster-data/master-0/data/instance:
uuid: "2fa1c2cc748a4cf69eff5f8953490615"
format_stamp: "Formatted at 2026-08-12 21:07:46 on dist-test-slave-vpvm"
server_key: "24af2819ddff6e3dd1c992233701bf96"
server_key_iv: "31e1237d47e8f4ff03f6939cd0f2b5a3"
server_key_version: "encryptionkey@0"
I20260812 21:07:46.602697 14450 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786568607354921-12282-0/minicluster-data/master-0/wal/instance:
uuid: "2fa1c2cc748a4cf69eff5f8953490615"
format_stamp: "Formatted at 2026-08-12 21:07:46 on dist-test-slave-vpvm"
server_key: "24af2819ddff6e3dd1c992233701bf96"
server_key_iv: "31e1237d47e8f4ff03f6939cd0f2b5a3"
server_key_version: "encryptionkey@0"
I20260812 21:07:46.604079 14450 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 21:07:46.604827 14476 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:07:46.604998 14450 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 21:07:46.605088 14450 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786568607354921-12282-0/minicluster-data/master-0/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786568607354921-12282-0/minicluster-data/master-0/wal
uuid: "2fa1c2cc748a4cf69eff5f8953490615"
format_stamp: "Formatted at 2026-08-12 21:07:46 on dist-test-slave-vpvm"
server_key: "24af2819ddff6e3dd1c992233701bf96"
server_key_iv: "31e1237d47e8f4ff03f6939cd0f2b5a3"
server_key_version: "encryptionkey@0"
I20260812 21:07:46.605159 14450 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786568607354921-12282-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786568607354921-12282-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786568607354921-12282-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:07:46.608685 14450 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:07:46.610884 14450 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:07:46.611042 14450 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:07:46.615165 14450 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35743
I20260812 21:07:46.615218 14566 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35743 every 8 connection(s)
I20260812 21:07:46.615545 14450 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
I20260812 21:07:46.616590 14567 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:07:46.619194 14567 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2fa1c2cc748a4cf69eff5f8953490615: Bootstrap starting.
I20260812 21:07:46.619802 14567 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2fa1c2cc748a4cf69eff5f8953490615: Neither blocks nor log segments found. Creating new log.
I20260812 21:07:46.620100 14567 log.cc:824] T 00000000000000000000000000000000 P 2fa1c2cc748a4cf69eff5f8953490615: Log is configured to *not* fsync() on all Append() calls
I20260812 21:07:46.620846 14567 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2fa1c2cc748a4cf69eff5f8953490615: No bootstrap required, opened a new log
I20260812 21:07:46.621138 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 14450
I20260812 21:07:46.621237 12282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786568607354921-12282-0/minicluster-data/master-0/wal/instance
I20260812 21:07:46.621402 12282 external_mini_cluster.cc:1428] Setting key 0e850233f7d54417fbe3b8091d2b95bc
I20260812 21:07:46.623040 14567 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2fa1c2cc748a4cf69eff5f8953490615 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fa1c2cc748a4cf69eff5f8953490615" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35743 } }
I20260812 21:07:46.623248 14567 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2fa1c2cc748a4cf69eff5f8953490615 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:07:46.623289 14567 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2fa1c2cc748a4cf69eff5f8953490615 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2fa1c2cc748a4cf69eff5f8953490615, State: Initialized, Role: FOLLOWER
I20260812 21:07:46.623471 14567 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2fa1c2cc748a4cf69eff5f8953490615 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fa1c2cc748a4cf69eff5f8953490615" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35743 } }
I20260812 21:07:46.623569 14567 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2fa1c2cc748a4cf69eff5f8953490615 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 21:07:46.623602 14567 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2fa1c2cc748a4cf69eff5f8953490615 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 21:07:46.623651 14567 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2fa1c2cc748a4cf69eff5f8953490615 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:07:46.624369 14574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:46.621892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52275 (local address 127.0.0.1:35743)
0812 21:07:46.622095 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0812 21:07:46.622105 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 21:07:46.622148 (+ 43us) server_negotiation.cc:407] Connection header received
0812 21:07:46.623013 (+ 865us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:07:46.623023 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:07:46.623061 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:07:46.623230 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:07:46.623839 (+ 609us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:07:46.623843 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:07:46.623844 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 21:07:46.623848 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:07:46.623897 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 21:07:46.623911 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:07:46.623988 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0812 21:07:46.624081 (+ 93us) server_negotiation.cc:299] Negotiation successful
0812 21:07:46.624126 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":104,"threads_started":1}
I20260812 21:07:46.624785 14567 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2fa1c2cc748a4cf69eff5f8953490615 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fa1c2cc748a4cf69eff5f8953490615" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35743 } }
I20260812 21:07:46.624891 14567 leader_election.cc:302] T 00000000000000000000000000000000 P 2fa1c2cc748a4cf69eff5f8953490615 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2fa1c2cc748a4cf69eff5f8953490615; no voters:
I20260812 21:07:46.625072 14567 leader_election.cc:288] T 00000000000000000000000000000000 P 2fa1c2cc748a4cf69eff5f8953490615 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 21:07:46.625113 14577 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2fa1c2cc748a4cf69eff5f8953490615 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:07:46.625279 14577 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2fa1c2cc748a4cf69eff5f8953490615 [term 1 LEADER]: Becoming Leader. State: Replica: 2fa1c2cc748a4cf69eff5f8953490615, State: Running, Role: LEADER
I20260812 21:07:46.625339 14567 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2fa1c2cc748a4cf69eff5f8953490615 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 21:07:46.625413 14577 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2fa1c2cc748a4cf69eff5f8953490615 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fa1c2cc748a4cf69eff5f8953490615" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35743 } }
I20260812 21:07:46.625447 12282 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 21:07:46.625875 14579 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2fa1c2cc748a4cf69eff5f8953490615 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2fa1c2cc748a4cf69eff5f8953490615" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fa1c2cc748a4cf69eff5f8953490615" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35743 } } }
I20260812 21:07:46.625996 14579 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2fa1c2cc748a4cf69eff5f8953490615 [sys.catalog]: This master's current role is: LEADER
I20260812 21:07:46.626194 14580 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2fa1c2cc748a4cf69eff5f8953490615 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2fa1c2cc748a4cf69eff5f8953490615. Latest consensus state: current_term: 1 leader_uuid: "2fa1c2cc748a4cf69eff5f8953490615" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fa1c2cc748a4cf69eff5f8953490615" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35743 } } }
I20260812 21:07:46.626309 14580 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2fa1c2cc748a4cf69eff5f8953490615 [sys.catalog]: This master's current role is: LEADER
W20260812 21:07:46.627177 14621 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2fa1c2cc748a4cf69eff5f8953490615: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 21:07:46.627223 14621 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 21:07:46.627288 14595 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 21:07:46.627467 14595 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 21:07:46.629104 14574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:46.627064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50823 (local address 127.0.0.1:35743)
0812 21:07:46.627098 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0812 21:07:46.627100 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:07:46.628201 (+ 1101us) server_negotiation.cc:407] Connection header received
0812 21:07:46.628248 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:07:46.628249 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:07:46.628254 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:07:46.628302 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:07:46.628879 (+ 577us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:07:46.628880 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:07:46.628881 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 21:07:46.628885 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:07:46.628902 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 21:07:46.628906 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:07:46.628963 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0812 21:07:46.629036 (+ 73us) server_negotiation.cc:299] Negotiation successful
0812 21:07:46.629047 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260812 21:07:46.630381 14595 catalog_manager.cc:1347] Generated new cluster ID: 103e9f81244e4b91b59ff074467b2906
I20260812 21:07:46.630450 14595 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 21:07:46.635448 14595 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 21:07:46.635617 14595 catalog_manager.cc:1504] Loading token signing keys...
I20260812 21:07:46.641569 14595 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2fa1c2cc748a4cf69eff5f8953490615: Generated new TSK 0
I20260812 21:07:46.641762 14595 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 21:07:46.643999 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 14450
2026-08-12T21:07:46Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1458 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786568607354921-12282-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 21:07:46 dist-test-slave-vpvm krb5kdc[14633](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 21:07:46 dist-test-slave-vpvm krb5kdc[14633](info): set up 2 sockets
Aug 12 21:07:46 dist-test-slave-vpvm krb5kdc[14633](info): commencing operation
krb5kdc: starting...
W20260812 21:07:47.829435 12282 mini_kdc.cc:121] Time spent starting KDC: real 1.154s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786568607354921-12282-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786568607354921-12282-0/krb5kdc/test-user.keytab.
Aug 12 21:07:47 dist-test-slave-vpvm krb5kdc[14633](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568867, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T21:07:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T21:07:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786568607354921-12282-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786568607354921-12282-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 21:07:47.957697 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786568607354921-12282-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786568607354921-12282-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786568607354921-12282-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36661
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786568607354921-12282-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45959
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36661
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786568607354921-12282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786568607354921-12282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786568607354921-12282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:07:48.042827 14661 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:07:48.043051 14661 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:07:48.043076 14661 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:07:48.044289 14661 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 21:07:48.044346 14661 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:07:48.044359 14661 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:07:48.044371 14661 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 21:07:48.044382 14661 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 21:07:48.045976 14661 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45959
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786568607354921-12282-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786568607354921-12282-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36661
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786568607354921-12282-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36661
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14661
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786568607354921-12282-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:07:48.046250 14661 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:07:48.046499 14661 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:07:48.051664 14671 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:07:48.051649 14666 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:07:48.051683 14668 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:07:48.051873 14661 server_base.cc:1034] running on GCE node
I20260812 21:07:48.052175 14661 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:07:48.052482 14661 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:07:48.053650 14661 hybrid_clock.cc:648] HybridClock initialized: now 1786568868053636 us; error 28 us; skew 500 ppm
Aug 12 21:07:48 dist-test-slave-vpvm krb5kdc[14633](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568868, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:07:48.056468 14661 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 21:07:48.056885 14661 webserver.cc:460] Webserver started at http://127.0.0.1:41821/ using document root <none> and password file <none>
I20260812 21:07:48.057106 14661 fs_manager.cc:359] Metadata directory not provided
I20260812 21:07:48.057147 14661 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:07:48.057243 14661 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:07:48.058357 14661 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786568607354921-12282-0/minicluster-data/master-0/data/instance:
uuid: "98bc369727ef488c89414730351f14ae"
format_stamp: "Formatted at 2026-08-12 21:07:48 on dist-test-slave-vpvm"
server_key: "c8d575d8d888c57d22ba793d8adc5e62"
server_key_iv: "e632f3c838ee1f53f124d92be61f9c3c"
server_key_version: "encryptionkey@0"
I20260812 21:07:48.058671 14661 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786568607354921-12282-0/minicluster-data/master-0/wal/instance:
uuid: "98bc369727ef488c89414730351f14ae"
format_stamp: "Formatted at 2026-08-12 21:07:48 on dist-test-slave-vpvm"
server_key: "c8d575d8d888c57d22ba793d8adc5e62"
server_key_iv: "e632f3c838ee1f53f124d92be61f9c3c"
server_key_version: "encryptionkey@0"
I20260812 21:07:48.059994 14661 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 21:07:48.060799 14679 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:07:48.060993 14661 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 21:07:48.061089 14661 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786568607354921-12282-0/minicluster-data/master-0/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786568607354921-12282-0/minicluster-data/master-0/wal
uuid: "98bc369727ef488c89414730351f14ae"
format_stamp: "Formatted at 2026-08-12 21:07:48 on dist-test-slave-vpvm"
server_key: "c8d575d8d888c57d22ba793d8adc5e62"
server_key_iv: "e632f3c838ee1f53f124d92be61f9c3c"
server_key_version: "encryptionkey@0"
I20260812 21:07:48.061152 14661 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786568607354921-12282-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786568607354921-12282-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786568607354921-12282-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:07:48.070046 14661 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:07:48.072281 14661 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:07:48.072476 14661 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:07:48.076829 14661 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36661
I20260812 21:07:48.076874 14766 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36661 every 8 connection(s)
I20260812 21:07:48.077225 14661 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
I20260812 21:07:48.078156 14768 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:07:48.080524 14768 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 98bc369727ef488c89414730351f14ae: Bootstrap starting.
I20260812 21:07:48.081120 14768 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 98bc369727ef488c89414730351f14ae: Neither blocks nor log segments found. Creating new log.
I20260812 21:07:48.081384 14768 log.cc:824] T 00000000000000000000000000000000 P 98bc369727ef488c89414730351f14ae: Log is configured to *not* fsync() on all Append() calls
I20260812 21:07:48.082273 14768 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 98bc369727ef488c89414730351f14ae: No bootstrap required, opened a new log
I20260812 21:07:48.083848 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 14661
I20260812 21:07:48.083966 12282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786568607354921-12282-0/minicluster-data/master-0/wal/instance
I20260812 21:07:48.084172 12282 external_mini_cluster.cc:1428] Setting key e2ff5ff2f2a2ef5708905317a0f67448
I20260812 21:07:48.084178 14768 raft_consensus.cc:348] T 00000000000000000000000000000000 P 98bc369727ef488c89414730351f14ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98bc369727ef488c89414730351f14ae" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36661 } }
I20260812 21:07:48.084372 14768 raft_consensus.cc:374] T 00000000000000000000000000000000 P 98bc369727ef488c89414730351f14ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:07:48.084414 14768 raft_consensus.cc:729] T 00000000000000000000000000000000 P 98bc369727ef488c89414730351f14ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98bc369727ef488c89414730351f14ae, State: Initialized, Role: FOLLOWER
I20260812 21:07:48.084583 14768 consensus_queue.cc:260] T 00000000000000000000000000000000 P 98bc369727ef488c89414730351f14ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98bc369727ef488c89414730351f14ae" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36661 } }
I20260812 21:07:48.084666 14768 raft_consensus.cc:388] T 00000000000000000000000000000000 P 98bc369727ef488c89414730351f14ae [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 21:07:48.084735 14768 raft_consensus.cc:482] T 00000000000000000000000000000000 P 98bc369727ef488c89414730351f14ae [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 21:07:48.084795 14768 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 98bc369727ef488c89414730351f14ae [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:07:48.085719 14768 raft_consensus.cc:504] T 00000000000000000000000000000000 P 98bc369727ef488c89414730351f14ae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98bc369727ef488c89414730351f14ae" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36661 } }
I20260812 21:07:48.085819 14768 leader_election.cc:302] T 00000000000000000000000000000000 P 98bc369727ef488c89414730351f14ae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 98bc369727ef488c89414730351f14ae; no voters:
I20260812 21:07:48.086016 14768 leader_election.cc:288] T 00000000000000000000000000000000 P 98bc369727ef488c89414730351f14ae [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 21:07:48.086037 14779 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 98bc369727ef488c89414730351f14ae [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:07:48.086154 14779 raft_consensus.cc:686] T 00000000000000000000000000000000 P 98bc369727ef488c89414730351f14ae [term 1 LEADER]: Becoming Leader. State: Replica: 98bc369727ef488c89414730351f14ae, State: Running, Role: LEADER
I20260812 21:07:48.086308 14768 sys_catalog.cc:564] T 00000000000000000000000000000000 P 98bc369727ef488c89414730351f14ae [sys.catalog]: configured and running, proceeding with master startup.
I20260812 21:07:48.086301 14779 consensus_queue.cc:237] T 00000000000000000000000000000000 P 98bc369727ef488c89414730351f14ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98bc369727ef488c89414730351f14ae" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36661 } }
I20260812 21:07:48.086715 14780 sys_catalog.cc:455] T 00000000000000000000000000000000 P 98bc369727ef488c89414730351f14ae [sys.catalog]: SysCatalogTable state changed. Reason: New leader 98bc369727ef488c89414730351f14ae. Latest consensus state: current_term: 1 leader_uuid: "98bc369727ef488c89414730351f14ae" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98bc369727ef488c89414730351f14ae" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36661 } } }
I20260812 21:07:48.086743 14781 sys_catalog.cc:455] T 00000000000000000000000000000000 P 98bc369727ef488c89414730351f14ae [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "98bc369727ef488c89414730351f14ae" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98bc369727ef488c89414730351f14ae" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36661 } } }
I20260812 21:07:48.086876 14780 sys_catalog.cc:458] T 00000000000000000000000000000000 P 98bc369727ef488c89414730351f14ae [sys.catalog]: This master's current role is: LEADER
I20260812 21:07:48.086886 14781 sys_catalog.cc:458] T 00000000000000000000000000000000 P 98bc369727ef488c89414730351f14ae [sys.catalog]: This master's current role is: LEADER
I20260812 21:07:48.087324 14794 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 21:07:48.087872 14794 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 21:07:48.089550 14794 catalog_manager.cc:1347] Generated new cluster ID: 33732cc9f2224c7db076e5cd69f22d56
I20260812 21:07:48.089605 14794 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 21:07:48 dist-test-slave-vpvm krb5kdc[14633](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568867, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260812 21:07:48.096205 14778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:48.084794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34109 (local address 127.0.0.1:36661)
0812 21:07:48.085055 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0812 21:07:48.085066 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 21:07:48.085094 (+ 28us) server_negotiation.cc:407] Connection header received
0812 21:07:48.085235 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:07:48.085241 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:07:48.085299 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:07:48.085441 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:07:48.086185 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:48.087600 (+ 1415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:48.088640 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:48.089145 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:48.089251 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:07:48.092324 (+ 3073us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:07:48.092329 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:07:48.092329 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:07:48.092343 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:07:48.094443 (+ 2100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:07:48.094839 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:07:48.094842 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:07:48.094843 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:07:48.094984 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:07:48.095176 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:07:48.095176 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:07:48.095177 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:07:48.095529 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:07:48.095718 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0812 21:07:48.095797 (+ 79us) server_negotiation.cc:299] Negotiation successful
0812 21:07:48.095956 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":181,"thread_start_us":111,"threads_started":1}
I20260812 21:07:48.097185 12282 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 21:07:48.105587 14778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:48.098768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52613 (local address 127.0.0.1:36661)
0812 21:07:48.098804 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0812 21:07:48.098805 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 21:07:48.098894 (+ 89us) server_negotiation.cc:407] Connection header received
0812 21:07:48.098935 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:07:48.098936 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:07:48.098947 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:07:48.099016 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:07:48.099578 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:48.100340 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:48.101297 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:48.101633 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:48.101699 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:07:48.103007 (+ 1308us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:07:48.103008 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:07:48.103008 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:07:48.103015 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:07:48.104526 (+ 1511us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:07:48.104869 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:07:48.104870 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:07:48.104870 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:07:48.104928 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:07:48.105096 (+ 168us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:07:48.105097 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:07:48.105097 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:07:48.105226 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:07:48.105323 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0812 21:07:48.105410 (+ 87us) server_negotiation.cc:299] Negotiation successful
0812 21:07:48.105488 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260812 21:07:48.110325 14794 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 21:07:48.111104 14794 catalog_manager.cc:1504] Loading token signing keys...
I20260812 21:07:48.118271 14794 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 98bc369727ef488c89414730351f14ae: Generated new TSK 0
I20260812 21:07:48.118486 14794 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 21:07:48.125603 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 14661
2026-08-12T21:07:48Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1481 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786568607354921-12282-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 21:07:48 dist-test-slave-vpvm krb5kdc[14822](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 21:07:48 dist-test-slave-vpvm krb5kdc[14822](info): set up 2 sockets
Aug 12 21:07:48 dist-test-slave-vpvm krb5kdc[14822](info): commencing operation
krb5kdc: starting...
W20260812 21:07:49.311712 12282 mini_kdc.cc:121] Time spent starting KDC: real 1.155s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786568607354921-12282-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786568607354921-12282-0/krb5kdc/test-user.keytab.
Aug 12 21:07:49 dist-test-slave-vpvm krb5kdc[14822](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568869, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T21:07:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T21:07:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786568607354921-12282-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786568607354921-12282-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 21:07:49.441345 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786568607354921-12282-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786568607354921-12282-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786568607354921-12282-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40785
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786568607354921-12282-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40855
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40785
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786568607354921-12282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786568607354921-12282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786568607354921-12282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:07:49.526561 14840 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:07:49.526783 14840 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:07:49.526818 14840 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:07:49.528106 14840 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 21:07:49.528183 14840 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:07:49.528211 14840 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:07:49.528235 14840 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 21:07:49.528266 14840 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 21:07:49.530332 14840 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40855
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786568607354921-12282-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786568607354921-12282-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40785
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786568607354921-12282-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40785
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14840
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786568607354921-12282-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:07:49.530616 14840 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:07:49.530889 14840 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:07:49.535986 14857 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:07:49.536090 14853 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:07:49.536108 14863 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:07:49.536267 14840 server_base.cc:1034] running on GCE node
I20260812 21:07:49.536576 14840 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:07:49.536866 14840 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:07:49.538041 14840 hybrid_clock.cc:648] HybridClock initialized: now 1786568869538019 us; error 38 us; skew 500 ppm
Aug 12 21:07:49 dist-test-slave-vpvm krb5kdc[14822](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568869, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:07:49.549290 14840 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 21:07:49.549908 14840 webserver.cc:460] Webserver started at http://127.0.0.1:35973/ using document root <none> and password file <none>
I20260812 21:07:49.550210 14840 fs_manager.cc:359] Metadata directory not provided
I20260812 21:07:49.550268 14840 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:07:49.550403 14840 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:07:49.551517 14840 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786568607354921-12282-0/minicluster-data/master-0/data/instance:
uuid: "bd5a6ee34cf541c3b381e6cde4858de3"
format_stamp: "Formatted at 2026-08-12 21:07:49 on dist-test-slave-vpvm"
server_key: "4254b8c5b16e7847654b1cf920c73bf3"
server_key_iv: "507e240284c169b237605dc98d9d1d87"
server_key_version: "encryptionkey@0"
I20260812 21:07:49.551877 14840 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786568607354921-12282-0/minicluster-data/master-0/wal/instance:
uuid: "bd5a6ee34cf541c3b381e6cde4858de3"
format_stamp: "Formatted at 2026-08-12 21:07:49 on dist-test-slave-vpvm"
server_key: "4254b8c5b16e7847654b1cf920c73bf3"
server_key_iv: "507e240284c169b237605dc98d9d1d87"
server_key_version: "encryptionkey@0"
I20260812 21:07:49.553263 14840 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 21:07:49.554239 14877 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:07:49.554392 14840 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 21:07:49.554492 14840 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786568607354921-12282-0/minicluster-data/master-0/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786568607354921-12282-0/minicluster-data/master-0/wal
uuid: "bd5a6ee34cf541c3b381e6cde4858de3"
format_stamp: "Formatted at 2026-08-12 21:07:49 on dist-test-slave-vpvm"
server_key: "4254b8c5b16e7847654b1cf920c73bf3"
server_key_iv: "507e240284c169b237605dc98d9d1d87"
server_key_version: "encryptionkey@0"
I20260812 21:07:49.554561 14840 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786568607354921-12282-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786568607354921-12282-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786568607354921-12282-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:07:49.571161 14840 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:07:49.573474 14840 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:07:49.573629 14840 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:07:49.577747 14840 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40785
I20260812 21:07:49.577790 14967 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40785 every 8 connection(s)
I20260812 21:07:49.578178 14840 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
I20260812 21:07:49.579051 14968 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:07:49.581457 14968 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bd5a6ee34cf541c3b381e6cde4858de3: Bootstrap starting.
I20260812 21:07:49.582104 14968 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bd5a6ee34cf541c3b381e6cde4858de3: Neither blocks nor log segments found. Creating new log.
I20260812 21:07:49.582376 14968 log.cc:824] T 00000000000000000000000000000000 P bd5a6ee34cf541c3b381e6cde4858de3: Log is configured to *not* fsync() on all Append() calls
I20260812 21:07:49.583123 14968 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bd5a6ee34cf541c3b381e6cde4858de3: No bootstrap required, opened a new log
I20260812 21:07:49.584807 14968 raft_consensus.cc:348] T 00000000000000000000000000000000 P bd5a6ee34cf541c3b381e6cde4858de3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd5a6ee34cf541c3b381e6cde4858de3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40785 } }
I20260812 21:07:49.584959 14968 raft_consensus.cc:374] T 00000000000000000000000000000000 P bd5a6ee34cf541c3b381e6cde4858de3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:07:49.584982 14968 raft_consensus.cc:729] T 00000000000000000000000000000000 P bd5a6ee34cf541c3b381e6cde4858de3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd5a6ee34cf541c3b381e6cde4858de3, State: Initialized, Role: FOLLOWER
I20260812 21:07:49.585124 14968 consensus_queue.cc:260] T 00000000000000000000000000000000 P bd5a6ee34cf541c3b381e6cde4858de3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd5a6ee34cf541c3b381e6cde4858de3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40785 } }
I20260812 21:07:49.585196 14968 raft_consensus.cc:388] T 00000000000000000000000000000000 P bd5a6ee34cf541c3b381e6cde4858de3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 21:07:49.585216 14968 raft_consensus.cc:482] T 00000000000000000000000000000000 P bd5a6ee34cf541c3b381e6cde4858de3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 21:07:49.585246 14968 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bd5a6ee34cf541c3b381e6cde4858de3 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:07:49.585808 14968 raft_consensus.cc:504] T 00000000000000000000000000000000 P bd5a6ee34cf541c3b381e6cde4858de3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd5a6ee34cf541c3b381e6cde4858de3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40785 } }
I20260812 21:07:49.585901 14968 leader_election.cc:302] T 00000000000000000000000000000000 P bd5a6ee34cf541c3b381e6cde4858de3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bd5a6ee34cf541c3b381e6cde4858de3; no voters:
I20260812 21:07:49.586108 14968 leader_election.cc:288] T 00000000000000000000000000000000 P bd5a6ee34cf541c3b381e6cde4858de3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 21:07:49.586162 14972 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bd5a6ee34cf541c3b381e6cde4858de3 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:07:49.586321 14972 raft_consensus.cc:686] T 00000000000000000000000000000000 P bd5a6ee34cf541c3b381e6cde4858de3 [term 1 LEADER]: Becoming Leader. State: Replica: bd5a6ee34cf541c3b381e6cde4858de3, State: Running, Role: LEADER
I20260812 21:07:49.586382 14968 sys_catalog.cc:564] T 00000000000000000000000000000000 P bd5a6ee34cf541c3b381e6cde4858de3 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 21:07:49.586426 14972 consensus_queue.cc:237] T 00000000000000000000000000000000 P bd5a6ee34cf541c3b381e6cde4858de3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd5a6ee34cf541c3b381e6cde4858de3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40785 } }
I20260812 21:07:49.586845 14975 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd5a6ee34cf541c3b381e6cde4858de3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bd5a6ee34cf541c3b381e6cde4858de3. Latest consensus state: current_term: 1 leader_uuid: "bd5a6ee34cf541c3b381e6cde4858de3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd5a6ee34cf541c3b381e6cde4858de3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40785 } } }
I20260812 21:07:49.586934 14975 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd5a6ee34cf541c3b381e6cde4858de3 [sys.catalog]: This master's current role is: LEADER
I20260812 21:07:49.587055 14974 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd5a6ee34cf541c3b381e6cde4858de3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bd5a6ee34cf541c3b381e6cde4858de3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd5a6ee34cf541c3b381e6cde4858de3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40785 } } }
I20260812 21:07:49.587191 14974 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd5a6ee34cf541c3b381e6cde4858de3 [sys.catalog]: This master's current role is: LEADER
I20260812 21:07:49.587373 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 14840
I20260812 21:07:49.587424 14983 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 21:07:49.587491 12282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786568607354921-12282-0/minicluster-data/master-0/wal/instance
I20260812 21:07:49.587673 12282 external_mini_cluster.cc:1428] Setting key 687e92ef9b44526d4f6136d30aed11d9
I20260812 21:07:49.587908 14983 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 21:07:49.589512 14983 catalog_manager.cc:1347] Generated new cluster ID: e8d87f1861c04d298d6a4c054265b429
I20260812 21:07:49.589571 14983 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 21:07:49.594657 14995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:49.588163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40889 (local address 127.0.0.1:40785)
0812 21:07:49.588902 (+ 739us) server_negotiation.cc:206] Beginning negotiation
0812 21:07:49.588912 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 21:07:49.588946 (+ 34us) server_negotiation.cc:407] Connection header received
0812 21:07:49.589075 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:07:49.589081 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:07:49.589138 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:07:49.589284 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:07:49.589559 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:49.591028 (+ 1469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:49.592152 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:49.592826 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:49.592957 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:07:49.593655 (+ 698us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:07:49.593660 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:07:49.593660 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 21:07:49.593883 (+ 223us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:07:49.593927 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 21:07:49.593943 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:07:49.594041 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0812 21:07:49.594102 (+ 61us) server_negotiation.cc:299] Negotiation successful
0812 21:07:49.594176 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":280,"thread_start_us":89,"threads_started":1}
I20260812 21:07:49.595729 12282 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 21:07:49.599300 14983 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 21:07:49.599757 14983 catalog_manager.cc:1504] Loading token signing keys...
I20260812 21:07:49.605177 14995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:49.598218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49665 (local address 127.0.0.1:40785)
0812 21:07:49.598254 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0812 21:07:49.598256 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:07:49.598875 (+ 619us) server_negotiation.cc:407] Connection header received
0812 21:07:49.599104 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:07:49.599105 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:07:49.599116 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:07:49.599163 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:07:49.602085 (+ 2922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:49.602874 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:49.603869 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:49.604207 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:49.604275 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:07:49.604914 (+ 639us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:07:49.604916 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:07:49.604916 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 21:07:49.604923 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:07:49.604940 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 21:07:49.604945 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:07:49.604999 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0812 21:07:49.605050 (+ 51us) server_negotiation.cc:299] Negotiation successful
0812 21:07:49.605098 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260812 21:07:49.615183 14983 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bd5a6ee34cf541c3b381e6cde4858de3: Generated new TSK 0
I20260812 21:07:49.615412 14983 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 21:07:49.618040 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 14840
2026-08-12T21:07:49Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1495 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786568607354921-12282-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 21:07:49 dist-test-slave-vpvm krb5kdc[15025](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 21:07:49 dist-test-slave-vpvm krb5kdc[15025](info): set up 2 sockets
Aug 12 21:07:49 dist-test-slave-vpvm krb5kdc[15025](info): commencing operation
krb5kdc: starting...
W20260812 21:07:50.804636 12282 mini_kdc.cc:121] Time spent starting KDC: real 1.153s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786568607354921-12282-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786568607354921-12282-0/krb5kdc/test-user.keytab.
Aug 12 21:07:50 dist-test-slave-vpvm krb5kdc[15025](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568870, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T21:07:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T21:07:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786568607354921-12282-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786568607354921-12282-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 21:07:50.932403 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786568607354921-12282-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786568607354921-12282-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786568607354921-12282-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44303
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786568607354921-12282-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43013
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44303
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786568607354921-12282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786568607354921-12282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786568607354921-12282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:07:51.017642 15046 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:07:51.017889 15046 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:07:51.017931 15046 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:07:51.019191 15046 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 21:07:51.019243 15046 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:07:51.019261 15046 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:07:51.019279 15046 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 21:07:51.019294 15046 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 21:07:51.020864 15046 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43013
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786568607354921-12282-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786568607354921-12282-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44303
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786568607354921-12282-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44303
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15046
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786568607354921-12282-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:07:51.021132 15046 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:07:51.021353 15046 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:07:51.026523 15053 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:07:51.026546 15054 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:07:51.026520 15057 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:07:51.026831 15046 server_base.cc:1034] running on GCE node
I20260812 21:07:51.027077 15046 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:07:51.027380 15046 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:07:51.028537 15046 hybrid_clock.cc:648] HybridClock initialized: now 1786568871028501 us; error 47 us; skew 500 ppm
Aug 12 21:07:51 dist-test-slave-vpvm krb5kdc[15025](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568871, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:07:51.031247 15046 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 21:07:51.031703 15046 webserver.cc:460] Webserver started at http://127.0.0.1:46393/ using document root <none> and password file <none>
I20260812 21:07:51.031917 15046 fs_manager.cc:359] Metadata directory not provided
I20260812 21:07:51.031976 15046 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:07:51.032109 15046 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:07:51.033208 15046 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786568607354921-12282-0/minicluster-data/master-0/data/instance:
uuid: "bf9f6137af7047309554b21bd054dbc6"
format_stamp: "Formatted at 2026-08-12 21:07:51 on dist-test-slave-vpvm"
server_key: "f10ee2d5e2fd7f4409c8f57de82cbf89"
server_key_iv: "2c7651918a5a233c5209da578cf5be6e"
server_key_version: "encryptionkey@0"
I20260812 21:07:51.033560 15046 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786568607354921-12282-0/minicluster-data/master-0/wal/instance:
uuid: "bf9f6137af7047309554b21bd054dbc6"
format_stamp: "Formatted at 2026-08-12 21:07:51 on dist-test-slave-vpvm"
server_key: "f10ee2d5e2fd7f4409c8f57de82cbf89"
server_key_iv: "2c7651918a5a233c5209da578cf5be6e"
server_key_version: "encryptionkey@0"
I20260812 21:07:51.034991 15046 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 21:07:51.035837 15069 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:07:51.036043 15046 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 21:07:51.036135 15046 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786568607354921-12282-0/minicluster-data/master-0/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786568607354921-12282-0/minicluster-data/master-0/wal
uuid: "bf9f6137af7047309554b21bd054dbc6"
format_stamp: "Formatted at 2026-08-12 21:07:51 on dist-test-slave-vpvm"
server_key: "f10ee2d5e2fd7f4409c8f57de82cbf89"
server_key_iv: "2c7651918a5a233c5209da578cf5be6e"
server_key_version: "encryptionkey@0"
I20260812 21:07:51.036199 15046 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786568607354921-12282-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786568607354921-12282-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786568607354921-12282-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:07:51.039722 15046 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:07:51.041843 15046 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:07:51.042016 15046 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:07:51.046046 15046 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44303
I20260812 21:07:51.046083 15160 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44303 every 8 connection(s)
I20260812 21:07:51.046424 15046 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
I20260812 21:07:51.047428 15162 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:07:51.048440 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 15046
I20260812 21:07:51.048561 12282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786568607354921-12282-0/minicluster-data/master-0/wal/instance
I20260812 21:07:51.048748 12282 external_mini_cluster.cc:1428] Setting key db24c8ffc8d7556e23e2df57c20695a3
I20260812 21:07:51.050997 15162 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bf9f6137af7047309554b21bd054dbc6: Bootstrap starting.
I20260812 21:07:51.051260 15165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:51.049323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58155 (local address 127.0.0.1:44303)
0812 21:07:51.049493 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0812 21:07:51.049501 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 21:07:51.049532 (+ 31us) server_negotiation.cc:407] Connection header received
0812 21:07:51.050039 (+ 507us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:07:51.050046 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:07:51.050074 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:07:51.050194 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:07:51.050834 (+ 640us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:07:51.050837 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:07:51.050837 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 21:07:51.050841 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:07:51.050880 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 21:07:51.050890 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:07:51.050944 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0812 21:07:51.051030 (+ 86us) server_negotiation.cc:299] Negotiation successful
0812 21:07:51.051067 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":66,"threads_started":1}
I20260812 21:07:51.051753 15162 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bf9f6137af7047309554b21bd054dbc6: Neither blocks nor log segments found. Creating new log.
I20260812 21:07:51.052050 15162 log.cc:824] T 00000000000000000000000000000000 P bf9f6137af7047309554b21bd054dbc6: Log is configured to *not* fsync() on all Append() calls
I20260812 21:07:51.052220 12282 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 21:07:51.052763 15162 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bf9f6137af7047309554b21bd054dbc6: No bootstrap required, opened a new log
I20260812 21:07:51.054551 15165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:51.053610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56041 (local address 127.0.0.1:44303)
0812 21:07:51.053648 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0812 21:07:51.053650 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:07:51.053710 (+ 60us) server_negotiation.cc:407] Connection header received
0812 21:07:51.053772 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:07:51.053772 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:07:51.053777 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:07:51.053817 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:07:51.054357 (+ 540us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:07:51.054358 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:07:51.054358 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 21:07:51.054362 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:07:51.054377 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 21:07:51.054381 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:07:51.054426 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0812 21:07:51.054476 (+ 50us) server_negotiation.cc:299] Negotiation successful
0812 21:07:51.054489 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 21:07:51.055061 15162 raft_consensus.cc:348] T 00000000000000000000000000000000 P bf9f6137af7047309554b21bd054dbc6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf9f6137af7047309554b21bd054dbc6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44303 } }
I20260812 21:07:51.055226 15162 raft_consensus.cc:374] T 00000000000000000000000000000000 P bf9f6137af7047309554b21bd054dbc6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:07:51.055249 15162 raft_consensus.cc:729] T 00000000000000000000000000000000 P bf9f6137af7047309554b21bd054dbc6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf9f6137af7047309554b21bd054dbc6, State: Initialized, Role: FOLLOWER
I20260812 21:07:51.055347 15162 consensus_queue.cc:260] T 00000000000000000000000000000000 P bf9f6137af7047309554b21bd054dbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf9f6137af7047309554b21bd054dbc6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44303 } }
I20260812 21:07:51.055429 15162 raft_consensus.cc:388] T 00000000000000000000000000000000 P bf9f6137af7047309554b21bd054dbc6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 21:07:51.055452 15162 raft_consensus.cc:482] T 00000000000000000000000000000000 P bf9f6137af7047309554b21bd054dbc6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 21:07:51.055487 15162 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bf9f6137af7047309554b21bd054dbc6 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:07:51.056049 15162 raft_consensus.cc:504] T 00000000000000000000000000000000 P bf9f6137af7047309554b21bd054dbc6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf9f6137af7047309554b21bd054dbc6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44303 } }
I20260812 21:07:51.056146 15162 leader_election.cc:302] T 00000000000000000000000000000000 P bf9f6137af7047309554b21bd054dbc6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bf9f6137af7047309554b21bd054dbc6; no voters:
I20260812 21:07:51.056301 15162 leader_election.cc:288] T 00000000000000000000000000000000 P bf9f6137af7047309554b21bd054dbc6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 21:07:51.056362 15178 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bf9f6137af7047309554b21bd054dbc6 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:07:51.056531 15162 sys_catalog.cc:564] T 00000000000000000000000000000000 P bf9f6137af7047309554b21bd054dbc6 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 21:07:51.056579 15178 raft_consensus.cc:686] T 00000000000000000000000000000000 P bf9f6137af7047309554b21bd054dbc6 [term 1 LEADER]: Becoming Leader. State: Replica: bf9f6137af7047309554b21bd054dbc6, State: Running, Role: LEADER
I20260812 21:07:51.056701 15178 consensus_queue.cc:237] T 00000000000000000000000000000000 P bf9f6137af7047309554b21bd054dbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf9f6137af7047309554b21bd054dbc6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44303 } }
I20260812 21:07:51.057040 15178 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf9f6137af7047309554b21bd054dbc6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bf9f6137af7047309554b21bd054dbc6. Latest consensus state: current_term: 1 leader_uuid: "bf9f6137af7047309554b21bd054dbc6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf9f6137af7047309554b21bd054dbc6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44303 } } }
I20260812 21:07:51.057164 15178 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf9f6137af7047309554b21bd054dbc6 [sys.catalog]: This master's current role is: LEADER
I20260812 21:07:51.057475 15192 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 21:07:51.057466 15179 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf9f6137af7047309554b21bd054dbc6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bf9f6137af7047309554b21bd054dbc6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf9f6137af7047309554b21bd054dbc6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44303 } } }
I20260812 21:07:51.057636 15179 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf9f6137af7047309554b21bd054dbc6 [sys.catalog]: This master's current role is: LEADER
I20260812 21:07:51.058054 15192 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 21:07:51.059648 15192 catalog_manager.cc:1347] Generated new cluster ID: c6c40514b302479fbb7f7262e7580f78
I20260812 21:07:51.059695 15192 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 21:07:51.068627 15192 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 21:07:51.068809 15192 catalog_manager.cc:1504] Loading token signing keys...
I20260812 21:07:51.086856 15192 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bf9f6137af7047309554b21bd054dbc6: Generated new TSK 0
I20260812 21:07:51.087081 15192 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 21:07:51.092259 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 15046
2026-08-12T21:07:51Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1470 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786568607354921-12282-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 21:07:51 dist-test-slave-vpvm krb5kdc[15219](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 21:07:51 dist-test-slave-vpvm krb5kdc[15219](info): set up 2 sockets
Aug 12 21:07:51 dist-test-slave-vpvm krb5kdc[15219](info): commencing operation
krb5kdc: starting...
W20260812 21:07:52.267207 12282 mini_kdc.cc:121] Time spent starting KDC: real 1.144s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786568607354921-12282-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786568607354921-12282-0/krb5kdc/test-user.keytab.
Aug 12 21:07:52 dist-test-slave-vpvm krb5kdc[15219](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568872, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T21:07:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T21:07:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786568607354921-12282-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786568607354921-12282-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 21:07:52.389025 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786568607354921-12282-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786568607354921-12282-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786568607354921-12282-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45299
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786568607354921-12282-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36099
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45299
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786568607354921-12282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786568607354921-12282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786568607354921-12282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:07:52.472158 15239 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:07:52.472390 15239 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:07:52.472414 15239 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:07:52.473635 15239 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 21:07:52.473691 15239 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:07:52.473704 15239 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:07:52.473716 15239 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 21:07:52.473729 15239 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 21:07:52.475315 15239 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36099
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786568607354921-12282-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786568607354921-12282-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45299
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786568607354921-12282-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45299
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15239
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786568607354921-12282-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:07:52.475572 15239 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:07:52.475804 15239 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:07:52.480798 15246 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:07:52.480847 15239 server_base.cc:1034] running on GCE node
W20260812 21:07:52.480811 15251 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:07:52.480815 15247 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:07:52.481321 15239 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:07:52.481634 15239 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:07:52.482786 15239 hybrid_clock.cc:648] HybridClock initialized: now 1786568872482775 us; error 30 us; skew 500 ppm
Aug 12 21:07:52 dist-test-slave-vpvm krb5kdc[15219](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568872, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:07:52.485314 15239 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 21:07:52.485746 15239 webserver.cc:460] Webserver started at http://127.0.0.1:38319/ using document root <none> and password file <none>
I20260812 21:07:52.485970 15239 fs_manager.cc:359] Metadata directory not provided
I20260812 21:07:52.486025 15239 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:07:52.486131 15239 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:07:52.487166 15239 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786568607354921-12282-0/minicluster-data/master-0/data/instance:
uuid: "3df36bb25aaf4356ae96bbd669e768f4"
format_stamp: "Formatted at 2026-08-12 21:07:52 on dist-test-slave-vpvm"
server_key: "bd37636f6cd32302746aad861534bc39"
server_key_iv: "031f8c5757bb76473984502e12950d2e"
server_key_version: "encryptionkey@0"
I20260812 21:07:52.487468 15239 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786568607354921-12282-0/minicluster-data/master-0/wal/instance:
uuid: "3df36bb25aaf4356ae96bbd669e768f4"
format_stamp: "Formatted at 2026-08-12 21:07:52 on dist-test-slave-vpvm"
server_key: "bd37636f6cd32302746aad861534bc39"
server_key_iv: "031f8c5757bb76473984502e12950d2e"
server_key_version: "encryptionkey@0"
I20260812 21:07:52.488749 15239 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 21:07:52.489768 15265 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:07:52.489990 15239 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 21:07:52.490096 15239 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786568607354921-12282-0/minicluster-data/master-0/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786568607354921-12282-0/minicluster-data/master-0/wal
uuid: "3df36bb25aaf4356ae96bbd669e768f4"
format_stamp: "Formatted at 2026-08-12 21:07:52 on dist-test-slave-vpvm"
server_key: "bd37636f6cd32302746aad861534bc39"
server_key_iv: "031f8c5757bb76473984502e12950d2e"
server_key_version: "encryptionkey@0"
I20260812 21:07:52.490175 15239 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786568607354921-12282-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786568607354921-12282-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786568607354921-12282-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:07:52.503479 15239 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:07:52.505728 15239 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:07:52.505875 15239 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:07:52.509825 15239 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45299
I20260812 21:07:52.509886 15354 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45299 every 8 connection(s)
I20260812 21:07:52.510248 15239 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
I20260812 21:07:52.511148 15355 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:07:52.513612 15355 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3df36bb25aaf4356ae96bbd669e768f4: Bootstrap starting.
I20260812 21:07:52.514257 15355 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3df36bb25aaf4356ae96bbd669e768f4: Neither blocks nor log segments found. Creating new log.
I20260812 21:07:52.514526 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 15239
I20260812 21:07:52.514534 15355 log.cc:824] T 00000000000000000000000000000000 P 3df36bb25aaf4356ae96bbd669e768f4: Log is configured to *not* fsync() on all Append() calls
I20260812 21:07:52.514648 12282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786568607354921-12282-0/minicluster-data/master-0/wal/instance
I20260812 21:07:52.514846 12282 external_mini_cluster.cc:1428] Setting key 971d494546f909285e4087ac3f1e9613
I20260812 21:07:52.515368 15355 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3df36bb25aaf4356ae96bbd669e768f4: No bootstrap required, opened a new log
I20260812 21:07:52.517449 15355 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3df36bb25aaf4356ae96bbd669e768f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3df36bb25aaf4356ae96bbd669e768f4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45299 } }
I20260812 21:07:52.517633 15355 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3df36bb25aaf4356ae96bbd669e768f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:07:52.517680 15355 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3df36bb25aaf4356ae96bbd669e768f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3df36bb25aaf4356ae96bbd669e768f4, State: Initialized, Role: FOLLOWER
I20260812 21:07:52.517810 15355 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3df36bb25aaf4356ae96bbd669e768f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3df36bb25aaf4356ae96bbd669e768f4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45299 } }
I20260812 21:07:52.517889 15355 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3df36bb25aaf4356ae96bbd669e768f4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 21:07:52.517926 15355 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3df36bb25aaf4356ae96bbd669e768f4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 21:07:52.518002 15355 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3df36bb25aaf4356ae96bbd669e768f4 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:07:52.518709 15355 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3df36bb25aaf4356ae96bbd669e768f4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3df36bb25aaf4356ae96bbd669e768f4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45299 } }
I20260812 21:07:52.518817 15355 leader_election.cc:302] T 00000000000000000000000000000000 P 3df36bb25aaf4356ae96bbd669e768f4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3df36bb25aaf4356ae96bbd669e768f4; no voters:
I20260812 21:07:52.518975 15355 leader_election.cc:288] T 00000000000000000000000000000000 P 3df36bb25aaf4356ae96bbd669e768f4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 21:07:52.519008 15369 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3df36bb25aaf4356ae96bbd669e768f4 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:07:52.519196 15369 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3df36bb25aaf4356ae96bbd669e768f4 [term 1 LEADER]: Becoming Leader. State: Replica: 3df36bb25aaf4356ae96bbd669e768f4, State: Running, Role: LEADER
I20260812 21:07:52.519212 15355 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3df36bb25aaf4356ae96bbd669e768f4 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 21:07:52.519290 15369 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3df36bb25aaf4356ae96bbd669e768f4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3df36bb25aaf4356ae96bbd669e768f4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45299 } }
I20260812 21:07:52.519660 15369 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3df36bb25aaf4356ae96bbd669e768f4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3df36bb25aaf4356ae96bbd669e768f4. Latest consensus state: current_term: 1 leader_uuid: "3df36bb25aaf4356ae96bbd669e768f4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3df36bb25aaf4356ae96bbd669e768f4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45299 } } }
I20260812 21:07:52.519716 15370 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3df36bb25aaf4356ae96bbd669e768f4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3df36bb25aaf4356ae96bbd669e768f4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3df36bb25aaf4356ae96bbd669e768f4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45299 } } }
I20260812 21:07:52.519796 15370 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3df36bb25aaf4356ae96bbd669e768f4 [sys.catalog]: This master's current role is: LEADER
I20260812 21:07:52.520015 15369 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3df36bb25aaf4356ae96bbd669e768f4 [sys.catalog]: This master's current role is: LEADER
I20260812 21:07:52.520179 15387 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 21:07:52.520938 15387 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 21:07:52.522985 15387 catalog_manager.cc:1347] Generated new cluster ID: 252e9a7e2a894349993b65cfa4a37c35
I20260812 21:07:52.523037 15387 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 21:07:52 dist-test-slave-vpvm krb5kdc[15219](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568872, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260812 21:07:52.527469 15364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:52.515427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.27:33401 (local address 127.0.0.1:45299)
0812 21:07:52.515611 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0812 21:07:52.515621 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 21:07:52.515655 (+ 34us) server_negotiation.cc:407] Connection header received
0812 21:07:52.516673 (+ 1018us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:07:52.516683 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:07:52.516750 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:07:52.516903 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:07:52.517505 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:52.519225 (+ 1720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:52.520305 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:52.520908 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:52.521009 (+ 101us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:07:52.523736 (+ 2727us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:07:52.523740 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:07:52.523741 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:07:52.523754 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:07:52.525732 (+ 1978us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:07:52.526184 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:07:52.526187 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:07:52.526188 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:07:52.526296 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:07:52.526499 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:07:52.526500 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:07:52.526500 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:07:52.526830 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:07:52.527015 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0812 21:07:52.527108 (+ 93us) server_negotiation.cc:299] Negotiation successful
0812 21:07:52.527220 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":95,"threads_started":1}
I20260812 21:07:52.528580 12282 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 21:07:52.529714 15387 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 21:07:52.530417 15387 catalog_manager.cc:1504] Loading token signing keys...
I20260812 21:07:52.536119 15387 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3df36bb25aaf4356ae96bbd669e768f4: Generated new TSK 0
I20260812 21:07:52.536299 15387 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 21:07:52.537032 15364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:52.529902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.27:58183 (local address 127.0.0.1:45299)
0812 21:07:52.529939 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0812 21:07:52.529941 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:07:52.530141 (+ 200us) server_negotiation.cc:407] Connection header received
0812 21:07:52.530199 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:07:52.530200 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:07:52.530211 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:07:52.530264 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:07:52.530877 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:52.531629 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:52.532660 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:52.532978 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:52.533029 (+ 51us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:07:52.534408 (+ 1379us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:07:52.534410 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:07:52.534410 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:07:52.534417 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:07:52.535895 (+ 1478us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:07:52.536347 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:07:52.536349 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:07:52.536349 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:07:52.536409 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:07:52.536623 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:07:52.536623 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:07:52.536624 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:07:52.536724 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:07:52.536797 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 21:07:52.536926 (+ 129us) server_negotiation.cc:299] Negotiation successful
0812 21:07:52.536975 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 21:07:52.538812 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 15239
W20260812 21:07:52.563671 15215 connection.cc:535] client connection to 127.0.0.1:45299 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:45299): Connection reset by peer (error 104)
2026-08-12T21:07:52Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1454 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786568607354921-12282-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 21:07:52 dist-test-slave-vpvm krb5kdc[15421](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 21:07:52 dist-test-slave-vpvm krb5kdc[15421](info): set up 2 sockets
Aug 12 21:07:52 dist-test-slave-vpvm krb5kdc[15421](info): commencing operation
krb5kdc: starting...
W20260812 21:07:53.732174 12282 mini_kdc.cc:121] Time spent starting KDC: real 1.155s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786568607354921-12282-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786568607354921-12282-0/krb5kdc/test-user.keytab.
Aug 12 21:07:53 dist-test-slave-vpvm krb5kdc[15421](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568873, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T21:07:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T21:07:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786568607354921-12282-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786568607354921-12282-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 21:07:53.854449 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786568607354921-12282-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786568607354921-12282-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786568607354921-12282-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42961
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786568607354921-12282-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42961
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786568607354921-12282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786568607354921-12282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786568607354921-12282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:07:53.937077 15452 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:07:53.937315 15452 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:07:53.937338 15452 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:07:53.938580 15452 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 21:07:53.938630 15452 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:07:53.938651 15452 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:07:53.938668 15452 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 21:07:53.938680 15452 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 21:07:53.940320 15452 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786568607354921-12282-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786568607354921-12282-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42961
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786568607354921-12282-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42961
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15452
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786568607354921-12282-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:07:53.940578 15452 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:07:53.940821 15452 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:07:53.945648 15460 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:07:53.945633 15461 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:07:53.945667 15452 server_base.cc:1034] running on GCE node
W20260812 21:07:53.945649 15464 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:07:53.946085 15452 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:07:53.946365 15452 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:07:53.947504 15452 hybrid_clock.cc:648] HybridClock initialized: now 1786568873947475 us; error 49 us; skew 500 ppm
Aug 12 21:07:53 dist-test-slave-vpvm krb5kdc[15421](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568873, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:07:53.950095 15452 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 21:07:53.950500 15452 webserver.cc:460] Webserver started at http://127.0.0.1:36061/ using document root <none> and password file <none>
I20260812 21:07:53.950713 15452 fs_manager.cc:359] Metadata directory not provided
I20260812 21:07:53.950753 15452 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:07:53.950847 15452 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:07:53.951891 15452 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786568607354921-12282-0/minicluster-data/master-0/data/instance:
uuid: "a76e9ae1f1764b3aa50fd19f6aadc6f3"
format_stamp: "Formatted at 2026-08-12 21:07:53 on dist-test-slave-vpvm"
server_key: "998ca147d7e06beb3dee90635610731d"
server_key_iv: "999c9e3164eb82a3625feab28a6eb8df"
server_key_version: "encryptionkey@0"
I20260812 21:07:53.952200 15452 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786568607354921-12282-0/minicluster-data/master-0/wal/instance:
uuid: "a76e9ae1f1764b3aa50fd19f6aadc6f3"
format_stamp: "Formatted at 2026-08-12 21:07:53 on dist-test-slave-vpvm"
server_key: "998ca147d7e06beb3dee90635610731d"
server_key_iv: "999c9e3164eb82a3625feab28a6eb8df"
server_key_version: "encryptionkey@0"
I20260812 21:07:53.953608 15452 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260812 21:07:53.954353 15474 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:07:53.954501 15452 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 21:07:53.954618 15452 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786568607354921-12282-0/minicluster-data/master-0/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786568607354921-12282-0/minicluster-data/master-0/wal
uuid: "a76e9ae1f1764b3aa50fd19f6aadc6f3"
format_stamp: "Formatted at 2026-08-12 21:07:53 on dist-test-slave-vpvm"
server_key: "998ca147d7e06beb3dee90635610731d"
server_key_iv: "999c9e3164eb82a3625feab28a6eb8df"
server_key_version: "encryptionkey@0"
I20260812 21:07:53.954691 15452 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786568607354921-12282-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786568607354921-12282-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786568607354921-12282-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:07:53.963678 15452 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:07:53.965827 15452 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:07:53.965993 15452 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:07:53.969810 15452 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42961
I20260812 21:07:53.969843 15579 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42961 every 8 connection(s)
I20260812 21:07:53.970196 15452 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
I20260812 21:07:53.971086 15580 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:07:53.973371 15580 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a76e9ae1f1764b3aa50fd19f6aadc6f3: Bootstrap starting.
I20260812 21:07:53.973942 15580 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a76e9ae1f1764b3aa50fd19f6aadc6f3: Neither blocks nor log segments found. Creating new log.
I20260812 21:07:53.974243 15580 log.cc:824] T 00000000000000000000000000000000 P a76e9ae1f1764b3aa50fd19f6aadc6f3: Log is configured to *not* fsync() on all Append() calls
I20260812 21:07:53.974942 15580 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a76e9ae1f1764b3aa50fd19f6aadc6f3: No bootstrap required, opened a new log
I20260812 21:07:53.976609 15580 raft_consensus.cc:348] T 00000000000000000000000000000000 P a76e9ae1f1764b3aa50fd19f6aadc6f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a76e9ae1f1764b3aa50fd19f6aadc6f3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42961 } }
I20260812 21:07:53.976752 15580 raft_consensus.cc:374] T 00000000000000000000000000000000 P a76e9ae1f1764b3aa50fd19f6aadc6f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:07:53.976785 15580 raft_consensus.cc:729] T 00000000000000000000000000000000 P a76e9ae1f1764b3aa50fd19f6aadc6f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a76e9ae1f1764b3aa50fd19f6aadc6f3, State: Initialized, Role: FOLLOWER
I20260812 21:07:53.976929 15580 consensus_queue.cc:260] T 00000000000000000000000000000000 P a76e9ae1f1764b3aa50fd19f6aadc6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a76e9ae1f1764b3aa50fd19f6aadc6f3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42961 } }
I20260812 21:07:53.977016 15580 raft_consensus.cc:388] T 00000000000000000000000000000000 P a76e9ae1f1764b3aa50fd19f6aadc6f3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 21:07:53.977057 15580 raft_consensus.cc:482] T 00000000000000000000000000000000 P a76e9ae1f1764b3aa50fd19f6aadc6f3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 21:07:53.977106 15580 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a76e9ae1f1764b3aa50fd19f6aadc6f3 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:07:53.977659 15580 raft_consensus.cc:504] T 00000000000000000000000000000000 P a76e9ae1f1764b3aa50fd19f6aadc6f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a76e9ae1f1764b3aa50fd19f6aadc6f3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42961 } }
I20260812 21:07:53.977756 15580 leader_election.cc:302] T 00000000000000000000000000000000 P a76e9ae1f1764b3aa50fd19f6aadc6f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a76e9ae1f1764b3aa50fd19f6aadc6f3; no voters:
I20260812 21:07:53.977910 15580 leader_election.cc:288] T 00000000000000000000000000000000 P a76e9ae1f1764b3aa50fd19f6aadc6f3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 21:07:53.977968 15585 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a76e9ae1f1764b3aa50fd19f6aadc6f3 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:07:53.978128 15585 raft_consensus.cc:686] T 00000000000000000000000000000000 P a76e9ae1f1764b3aa50fd19f6aadc6f3 [term 1 LEADER]: Becoming Leader. State: Replica: a76e9ae1f1764b3aa50fd19f6aadc6f3, State: Running, Role: LEADER
I20260812 21:07:53.978186 15580 sys_catalog.cc:564] T 00000000000000000000000000000000 P a76e9ae1f1764b3aa50fd19f6aadc6f3 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 21:07:53.978221 15585 consensus_queue.cc:237] T 00000000000000000000000000000000 P a76e9ae1f1764b3aa50fd19f6aadc6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a76e9ae1f1764b3aa50fd19f6aadc6f3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42961 } }
I20260812 21:07:53.978595 15587 sys_catalog.cc:455] T 00000000000000000000000000000000 P a76e9ae1f1764b3aa50fd19f6aadc6f3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a76e9ae1f1764b3aa50fd19f6aadc6f3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a76e9ae1f1764b3aa50fd19f6aadc6f3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42961 } } }
I20260812 21:07:53.978693 15587 sys_catalog.cc:458] T 00000000000000000000000000000000 P a76e9ae1f1764b3aa50fd19f6aadc6f3 [sys.catalog]: This master's current role is: LEADER
I20260812 21:07:53.978606 15591 sys_catalog.cc:455] T 00000000000000000000000000000000 P a76e9ae1f1764b3aa50fd19f6aadc6f3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a76e9ae1f1764b3aa50fd19f6aadc6f3. Latest consensus state: current_term: 1 leader_uuid: "a76e9ae1f1764b3aa50fd19f6aadc6f3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a76e9ae1f1764b3aa50fd19f6aadc6f3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42961 } } }
I20260812 21:07:53.978766 15591 sys_catalog.cc:458] T 00000000000000000000000000000000 P a76e9ae1f1764b3aa50fd19f6aadc6f3 [sys.catalog]: This master's current role is: LEADER
I20260812 21:07:53.978930 15594 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 21:07:53.979419 15594 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 21:07:53.979700 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 15452
I20260812 21:07:53.979815 12282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786568607354921-12282-0/minicluster-data/master-0/wal/instance
I20260812 21:07:53.979982 12282 external_mini_cluster.cc:1428] Setting key b3a68b6dfdca41c117c4ba497c3a5937
I20260812 21:07:53.981370 15594 catalog_manager.cc:1347] Generated new cluster ID: f7de341b82fd4723923c097d1c11b7c8
I20260812 21:07:53.981424 15594 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 21:07:53.986280 15618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:53.980423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.27:37135 (local address 127.0.0.1:42961)
0812 21:07:53.980602 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0812 21:07:53.980611 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 21:07:53.980654 (+ 43us) server_negotiation.cc:407] Connection header received
0812 21:07:53.980836 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:07:53.980843 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:07:53.980897 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:07:53.981001 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:07:53.981247 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:53.982768 (+ 1521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:53.983908 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:53.984573 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:53.984676 (+ 103us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:07:53.985433 (+ 757us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:07:53.985438 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:07:53.985438 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 21:07:53.985650 (+ 212us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:07:53.985691 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 21:07:53.985705 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:07:53.985782 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0812 21:07:53.985892 (+ 110us) server_negotiation.cc:299] Negotiation successful
0812 21:07:53.986054 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":71,"threads_started":1}
I20260812 21:07:53.987278 12282 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 21:07:53.988955 15594 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 21:07:53.989444 15594 catalog_manager.cc:1504] Loading token signing keys...
I20260812 21:07:53.991432 15594 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a76e9ae1f1764b3aa50fd19f6aadc6f3: Generated new TSK 0
I20260812 21:07:53.991602 15594 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 21:07:53.994560 15618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:53.988854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.27:44229 (local address 127.0.0.1:42961)
0812 21:07:53.988885 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0812 21:07:53.988888 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 21:07:53.991156 (+ 2268us) server_negotiation.cc:407] Connection header received
0812 21:07:53.991211 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:07:53.991212 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:07:53.991226 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:07:53.991287 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:07:53.991771 (+ 484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:53.992592 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:53.993288 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:53.993613 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:53.993673 (+ 60us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:07:53.994329 (+ 656us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:07:53.994330 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:07:53.994330 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 21:07:53.994335 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:07:53.994350 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 21:07:53.994354 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:07:53.994417 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0812 21:07:53.994469 (+ 52us) server_negotiation.cc:299] Negotiation successful
0812 21:07:53.994482 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260812 21:07:53.996362 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 15452
W20260812 21:07:54.011158 15419 connection.cc:535] client connection to 127.0.0.1:42961 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42961): Connection reset by peer (error 104)
2026-08-12T21:07:54Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1447 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786568607354921-12282-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 21:07:54 dist-test-slave-vpvm krb5kdc[15639](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 21:07:54 dist-test-slave-vpvm krb5kdc[15639](info): set up 2 sockets
Aug 12 21:07:54 dist-test-slave-vpvm krb5kdc[15639](info): commencing operation
krb5kdc: starting...
W20260812 21:07:55.180228 12282 mini_kdc.cc:121] Time spent starting KDC: real 1.155s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786568607354921-12282-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786568607354921-12282-0/krb5kdc/test-user.keytab.
Aug 12 21:07:55 dist-test-slave-vpvm krb5kdc[15639](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568875, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T21:07:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T21:07:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786568607354921-12282-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786568607354921-12282-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 21:07:55.305529 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786568607354921-12282-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786568607354921-12282-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786568607354921-12282-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46125
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786568607354921-12282-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36545
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46125
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786568607354921-12282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786568607354921-12282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786568607354921-12282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:07:55.387917 15665 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:07:55.388154 15665 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:07:55.388187 15665 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:07:55.389346 15665 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 21:07:55.389387 15665 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:07:55.389406 15665 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:07:55.389423 15665 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 21:07:55.389438 15665 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 21:07:55.390978 15665 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36545
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786568607354921-12282-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786568607354921-12282-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46125
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786568607354921-12282-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46125
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15665
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786568607354921-12282-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:07:55.391239 15665 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:07:55.391495 15665 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:07:55.396471 15674 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:07:55.396435 15665 server_base.cc:1034] running on GCE node
W20260812 21:07:55.396479 15676 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:07:55.396441 15673 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:07:55.396873 15665 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:07:55.397166 15665 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:07:55.398307 15665 hybrid_clock.cc:648] HybridClock initialized: now 1786568875398291 us; error 30 us; skew 500 ppm
Aug 12 21:07:55 dist-test-slave-vpvm krb5kdc[15639](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568875, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:07:55.406343 15665 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 21:07:55.406810 15665 webserver.cc:460] Webserver started at http://127.0.0.1:41865/ using document root <none> and password file <none>
I20260812 21:07:55.407007 15665 fs_manager.cc:359] Metadata directory not provided
I20260812 21:07:55.407049 15665 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:07:55.407150 15665 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:07:55.408223 15665 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786568607354921-12282-0/minicluster-data/master-0/data/instance:
uuid: "e07ea3d6ad094af1a456eea579a8d329"
format_stamp: "Formatted at 2026-08-12 21:07:55 on dist-test-slave-vpvm"
server_key: "4849733470204b203e347e8cdc9f0f77"
server_key_iv: "4767170c9268c17e0f8b9f2361a4b819"
server_key_version: "encryptionkey@0"
I20260812 21:07:55.408542 15665 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786568607354921-12282-0/minicluster-data/master-0/wal/instance:
uuid: "e07ea3d6ad094af1a456eea579a8d329"
format_stamp: "Formatted at 2026-08-12 21:07:55 on dist-test-slave-vpvm"
server_key: "4849733470204b203e347e8cdc9f0f77"
server_key_iv: "4767170c9268c17e0f8b9f2361a4b819"
server_key_version: "encryptionkey@0"
I20260812 21:07:55.409816 15665 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260812 21:07:55.410601 15688 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:07:55.410753 15665 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 21:07:55.410840 15665 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786568607354921-12282-0/minicluster-data/master-0/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786568607354921-12282-0/minicluster-data/master-0/wal
uuid: "e07ea3d6ad094af1a456eea579a8d329"
format_stamp: "Formatted at 2026-08-12 21:07:55 on dist-test-slave-vpvm"
server_key: "4849733470204b203e347e8cdc9f0f77"
server_key_iv: "4767170c9268c17e0f8b9f2361a4b819"
server_key_version: "encryptionkey@0"
I20260812 21:07:55.410899 15665 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786568607354921-12282-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786568607354921-12282-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786568607354921-12282-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:07:55.414194 15665 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:07:55.416262 15665 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:07:55.416368 15665 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:07:55.420295 15665 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46125
I20260812 21:07:55.420336 15790 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46125 every 8 connection(s)
I20260812 21:07:55.420647 15665 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
I20260812 21:07:55.420854 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 15665
I20260812 21:07:55.420976 12282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786568607354921-12282-0/minicluster-data/master-0/wal/instance
I20260812 21:07:55.421182 12282 external_mini_cluster.cc:1428] Setting key 6263591e5a0a610a141e54a6f6b5255d
I20260812 21:07:55.421864 15792 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:07:55.423723 15798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:55.421750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.27:34813 (local address 127.0.0.1:46125)
0812 21:07:55.421965 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0812 21:07:55.421977 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 21:07:55.422010 (+ 33us) server_negotiation.cc:407] Connection header received
0812 21:07:55.422470 (+ 460us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:07:55.422476 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:07:55.422501 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:07:55.422626 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:07:55.423298 (+ 672us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:07:55.423301 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:07:55.423301 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 21:07:55.423305 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:07:55.423345 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 21:07:55.423355 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:07:55.423410 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0812 21:07:55.423471 (+ 61us) server_negotiation.cc:299] Negotiation successful
0812 21:07:55.423514 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":89,"threads_started":1}
I20260812 21:07:55.425000 15792 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e07ea3d6ad094af1a456eea579a8d329: Bootstrap starting.
I20260812 21:07:55.425276 12282 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 21:07:55.425668 15792 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e07ea3d6ad094af1a456eea579a8d329: Neither blocks nor log segments found. Creating new log.
I20260812 21:07:55.425930 15792 log.cc:824] T 00000000000000000000000000000000 P e07ea3d6ad094af1a456eea579a8d329: Log is configured to *not* fsync() on all Append() calls
I20260812 21:07:55.426743 15792 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e07ea3d6ad094af1a456eea579a8d329: No bootstrap required, opened a new log
I20260812 21:07:55.427661 15798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:55.426671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.27:38679 (local address 127.0.0.1:46125)
0812 21:07:55.426711 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0812 21:07:55.426713 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:07:55.426790 (+ 77us) server_negotiation.cc:407] Connection header received
0812 21:07:55.426881 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:07:55.426882 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:07:55.426887 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:07:55.426930 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:07:55.427452 (+ 522us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:07:55.427453 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:07:55.427453 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 21:07:55.427457 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:07:55.427485 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 21:07:55.427489 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:07:55.427550 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0812 21:07:55.427587 (+ 37us) server_negotiation.cc:299] Negotiation successful
0812 21:07:55.427599 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 21:07:55.428825 15792 raft_consensus.cc:348] T 00000000000000000000000000000000 P e07ea3d6ad094af1a456eea579a8d329 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e07ea3d6ad094af1a456eea579a8d329" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46125 } }
I20260812 21:07:55.428963 15792 raft_consensus.cc:374] T 00000000000000000000000000000000 P e07ea3d6ad094af1a456eea579a8d329 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:07:55.428985 15792 raft_consensus.cc:729] T 00000000000000000000000000000000 P e07ea3d6ad094af1a456eea579a8d329 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e07ea3d6ad094af1a456eea579a8d329, State: Initialized, Role: FOLLOWER
I20260812 21:07:55.429077 15792 consensus_queue.cc:260] T 00000000000000000000000000000000 P e07ea3d6ad094af1a456eea579a8d329 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e07ea3d6ad094af1a456eea579a8d329" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46125 } }
I20260812 21:07:55.429164 15792 raft_consensus.cc:388] T 00000000000000000000000000000000 P e07ea3d6ad094af1a456eea579a8d329 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 21:07:55.429200 15792 raft_consensus.cc:482] T 00000000000000000000000000000000 P e07ea3d6ad094af1a456eea579a8d329 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 21:07:55.429250 15792 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e07ea3d6ad094af1a456eea579a8d329 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:07:55.429787 15792 raft_consensus.cc:504] T 00000000000000000000000000000000 P e07ea3d6ad094af1a456eea579a8d329 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e07ea3d6ad094af1a456eea579a8d329" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46125 } }
I20260812 21:07:55.429874 15792 leader_election.cc:302] T 00000000000000000000000000000000 P e07ea3d6ad094af1a456eea579a8d329 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e07ea3d6ad094af1a456eea579a8d329; no voters:
I20260812 21:07:55.430037 15792 leader_election.cc:288] T 00000000000000000000000000000000 P e07ea3d6ad094af1a456eea579a8d329 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 21:07:55.430081 15812 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e07ea3d6ad094af1a456eea579a8d329 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:07:55.430240 15812 raft_consensus.cc:686] T 00000000000000000000000000000000 P e07ea3d6ad094af1a456eea579a8d329 [term 1 LEADER]: Becoming Leader. State: Replica: e07ea3d6ad094af1a456eea579a8d329, State: Running, Role: LEADER
I20260812 21:07:55.430258 15792 sys_catalog.cc:564] T 00000000000000000000000000000000 P e07ea3d6ad094af1a456eea579a8d329 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 21:07:55.430404 15812 consensus_queue.cc:237] T 00000000000000000000000000000000 P e07ea3d6ad094af1a456eea579a8d329 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e07ea3d6ad094af1a456eea579a8d329" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46125 } }
I20260812 21:07:55.430768 15812 sys_catalog.cc:455] T 00000000000000000000000000000000 P e07ea3d6ad094af1a456eea579a8d329 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e07ea3d6ad094af1a456eea579a8d329. Latest consensus state: current_term: 1 leader_uuid: "e07ea3d6ad094af1a456eea579a8d329" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e07ea3d6ad094af1a456eea579a8d329" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46125 } } }
I20260812 21:07:55.430845 15812 sys_catalog.cc:458] T 00000000000000000000000000000000 P e07ea3d6ad094af1a456eea579a8d329 [sys.catalog]: This master's current role is: LEADER
I20260812 21:07:55.430804 15816 sys_catalog.cc:455] T 00000000000000000000000000000000 P e07ea3d6ad094af1a456eea579a8d329 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e07ea3d6ad094af1a456eea579a8d329" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e07ea3d6ad094af1a456eea579a8d329" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46125 } } }
I20260812 21:07:55.430891 15816 sys_catalog.cc:458] T 00000000000000000000000000000000 P e07ea3d6ad094af1a456eea579a8d329 [sys.catalog]: This master's current role is: LEADER
I20260812 21:07:55.431054 15835 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 21:07:55.431685 15835 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 21:07:55.433252 15835 catalog_manager.cc:1347] Generated new cluster ID: 36d28dbc1db940f7833d551c437ccf84
I20260812 21:07:55.433311 15835 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 21:07:55.457247 15835 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 21:07:55.457438 15835 catalog_manager.cc:1504] Loading token signing keys...
I20260812 21:07:55.466382 15835 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e07ea3d6ad094af1a456eea579a8d329: Generated new TSK 0
I20260812 21:07:55.466599 15835 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 21:07:55.472324 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 15665
2026-08-12T21:07:55Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1470 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786568607354921-12282-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 21:07:55 dist-test-slave-vpvm krb5kdc[15861](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 21:07:55 dist-test-slave-vpvm krb5kdc[15861](info): set up 2 sockets
Aug 12 21:07:55 dist-test-slave-vpvm krb5kdc[15861](info): commencing operation
krb5kdc: starting...
W20260812 21:07:56.654335 12282 mini_kdc.cc:121] Time spent starting KDC: real 1.159s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786568607354921-12282-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786568607354921-12282-0/krb5kdc/test-user.keytab.
Aug 12 21:07:56 dist-test-slave-vpvm krb5kdc[15861](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568876, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T21:07:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T21:07:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786568607354921-12282-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786568607354921-12282-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 21:07:56.780491 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786568607354921-12282-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786568607354921-12282-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786568607354921-12282-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43711
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786568607354921-12282-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43711
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786568607354921-12282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786568607354921-12282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786568607354921-12282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:07:56.862856 15884 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:07:56.863065 15884 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:07:56.863087 15884 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:07:56.864212 15884 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 21:07:56.864250 15884 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:07:56.864264 15884 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:07:56.864274 15884 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 21:07:56.864284 15884 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 21:07:56.865784 15884 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786568607354921-12282-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786568607354921-12282-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43711
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786568607354921-12282-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43711
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15884
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786568607354921-12282-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:07:56.866034 15884 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:07:56.866235 15884 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:07:56.871340 15894 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:07:56.871376 15893 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:07:56.871344 15897 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:07:56.871501 15884 server_base.cc:1034] running on GCE node
I20260812 21:07:56.871833 15884 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:07:56.872123 15884 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:07:56.873263 15884 hybrid_clock.cc:648] HybridClock initialized: now 1786568876873246 us; error 29 us; skew 500 ppm
Aug 12 21:07:56 dist-test-slave-vpvm krb5kdc[15861](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568876, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:07:56.875751 15884 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 21:07:56.876158 15884 webserver.cc:460] Webserver started at http://127.0.0.1:33681/ using document root <none> and password file <none>
I20260812 21:07:56.876365 15884 fs_manager.cc:359] Metadata directory not provided
I20260812 21:07:56.876405 15884 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:07:56.876510 15884 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:07:56.877583 15884 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786568607354921-12282-0/minicluster-data/master-0/data/instance:
uuid: "91178727cf734118870654aa0db441a9"
format_stamp: "Formatted at 2026-08-12 21:07:56 on dist-test-slave-vpvm"
server_key: "9aa8daaef4b9266cf4d9b66c9644d924"
server_key_iv: "c9aebba84eb87e04c3b254de8fbfb860"
server_key_version: "encryptionkey@0"
I20260812 21:07:56.877877 15884 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786568607354921-12282-0/minicluster-data/master-0/wal/instance:
uuid: "91178727cf734118870654aa0db441a9"
format_stamp: "Formatted at 2026-08-12 21:07:56 on dist-test-slave-vpvm"
server_key: "9aa8daaef4b9266cf4d9b66c9644d924"
server_key_iv: "c9aebba84eb87e04c3b254de8fbfb860"
server_key_version: "encryptionkey@0"
I20260812 21:07:56.879159 15884 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 21:07:56.879851 15914 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:07:56.880012 15884 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 21:07:56.880101 15884 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786568607354921-12282-0/minicluster-data/master-0/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786568607354921-12282-0/minicluster-data/master-0/wal
uuid: "91178727cf734118870654aa0db441a9"
format_stamp: "Formatted at 2026-08-12 21:07:56 on dist-test-slave-vpvm"
server_key: "9aa8daaef4b9266cf4d9b66c9644d924"
server_key_iv: "c9aebba84eb87e04c3b254de8fbfb860"
server_key_version: "encryptionkey@0"
I20260812 21:07:56.880168 15884 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786568607354921-12282-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786568607354921-12282-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786568607354921-12282-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:07:56.898259 15884 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:07:56.900460 15884 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:07:56.900581 15884 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:07:56.904551 15884 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43711
I20260812 21:07:56.904623 16010 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43711 every 8 connection(s)
I20260812 21:07:56.904958 15884 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
I20260812 21:07:56.905900 16012 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:07:56.906162 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 15884
I20260812 21:07:56.906255 12282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786568607354921-12282-0/minicluster-data/master-0/wal/instance
I20260812 21:07:56.906405 12282 external_mini_cluster.cc:1428] Setting key b082f084de930c46def39c46bc6ef30e
I20260812 21:07:56.909009 16012 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 91178727cf734118870654aa0db441a9: Bootstrap starting.
I20260812 21:07:56.909739 16012 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 91178727cf734118870654aa0db441a9: Neither blocks nor log segments found. Creating new log.
I20260812 21:07:56.910110 16012 log.cc:824] T 00000000000000000000000000000000 P 91178727cf734118870654aa0db441a9: Log is configured to *not* fsync() on all Append() calls
I20260812 21:07:56.910805 16012 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 91178727cf734118870654aa0db441a9: No bootstrap required, opened a new log
I20260812 21:07:56.912943 16012 raft_consensus.cc:348] T 00000000000000000000000000000000 P 91178727cf734118870654aa0db441a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91178727cf734118870654aa0db441a9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43711 } }
I20260812 21:07:56.913093 16012 raft_consensus.cc:374] T 00000000000000000000000000000000 P 91178727cf734118870654aa0db441a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:07:56.913126 16012 raft_consensus.cc:729] T 00000000000000000000000000000000 P 91178727cf734118870654aa0db441a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91178727cf734118870654aa0db441a9, State: Initialized, Role: FOLLOWER
I20260812 21:07:56.913257 16012 consensus_queue.cc:260] T 00000000000000000000000000000000 P 91178727cf734118870654aa0db441a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91178727cf734118870654aa0db441a9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43711 } }
I20260812 21:07:56.913354 16012 raft_consensus.cc:388] T 00000000000000000000000000000000 P 91178727cf734118870654aa0db441a9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
Aug 12 21:07:56 dist-test-slave-vpvm krb5kdc[15861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568876, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260812 21:07:56.913398 16012 raft_consensus.cc:482] T 00000000000000000000000000000000 P 91178727cf734118870654aa0db441a9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 21:07:56.913450 16012 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 91178727cf734118870654aa0db441a9 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:07:56.914216 16012 raft_consensus.cc:504] T 00000000000000000000000000000000 P 91178727cf734118870654aa0db441a9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91178727cf734118870654aa0db441a9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43711 } }
I20260812 21:07:56.914312 16012 leader_election.cc:302] T 00000000000000000000000000000000 P 91178727cf734118870654aa0db441a9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 91178727cf734118870654aa0db441a9; no voters:
I20260812 21:07:56.914496 16012 leader_election.cc:288] T 00000000000000000000000000000000 P 91178727cf734118870654aa0db441a9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 21:07:56.914542 16023 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 91178727cf734118870654aa0db441a9 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:07:56.914686 16023 raft_consensus.cc:686] T 00000000000000000000000000000000 P 91178727cf734118870654aa0db441a9 [term 1 LEADER]: Becoming Leader. State: Replica: 91178727cf734118870654aa0db441a9, State: Running, Role: LEADER
I20260812 21:07:56.914778 16012 sys_catalog.cc:564] T 00000000000000000000000000000000 P 91178727cf734118870654aa0db441a9 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 21:07:56.914821 16023 consensus_queue.cc:237] T 00000000000000000000000000000000 P 91178727cf734118870654aa0db441a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91178727cf734118870654aa0db441a9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43711 } }
I20260812 21:07:56.915540 16024 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91178727cf734118870654aa0db441a9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 91178727cf734118870654aa0db441a9. Latest consensus state: current_term: 1 leader_uuid: "91178727cf734118870654aa0db441a9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91178727cf734118870654aa0db441a9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43711 } } }
I20260812 21:07:56.915545 16027 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91178727cf734118870654aa0db441a9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "91178727cf734118870654aa0db441a9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91178727cf734118870654aa0db441a9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43711 } } }
I20260812 21:07:56.915684 16027 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91178727cf734118870654aa0db441a9 [sys.catalog]: This master's current role is: LEADER
I20260812 21:07:56.915678 16024 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91178727cf734118870654aa0db441a9 [sys.catalog]: This master's current role is: LEADER
I20260812 21:07:56.916322 16041 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 21:07:56.917026 16041 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 21:07:56.917833 16018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:56.906935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.27:40215 (local address 127.0.0.1:43711)
0812 21:07:56.907140 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0812 21:07:56.907151 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 21:07:56.907187 (+ 36us) server_negotiation.cc:407] Connection header received
0812 21:07:56.907739 (+ 552us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:07:56.907749 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:07:56.907821 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:07:56.907982 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:07:56.908595 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:56.909981 (+ 1386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:56.910784 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:56.911377 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:56.911512 (+ 135us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:07:56.913998 (+ 2486us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:07:56.914002 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:07:56.914002 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:07:56.914013 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:07:56.916063 (+ 2050us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:07:56.916517 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:07:56.916520 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:07:56.916520 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:07:56.916624 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:07:56.916877 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:07:56.916878 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:07:56.916878 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:07:56.917213 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:07:56.917381 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0812 21:07:56.917522 (+ 141us) server_negotiation.cc:299] Negotiation successful
0812 21:07:56.917630 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":97,"threads_started":1}
I20260812 21:07:56.918596 16041 catalog_manager.cc:1347] Generated new cluster ID: 78af38771c0d4839a6578004e0026f0e
I20260812 21:07:56.918661 16041 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 21:07:56.918859 12282 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 21:07:56.933077 16018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:56.925278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.27:60995 (local address 127.0.0.1:43711)
0812 21:07:56.925334 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0812 21:07:56.925336 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:07:56.925535 (+ 199us) server_negotiation.cc:407] Connection header received
0812 21:07:56.925608 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:07:56.925609 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:07:56.925621 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:07:56.925669 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:07:56.926358 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:56.927159 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:56.928249 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:56.928625 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:56.928699 (+ 74us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:07:56.930258 (+ 1559us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:07:56.930259 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:07:56.930260 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:07:56.930268 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:07:56.931853 (+ 1585us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:07:56.932269 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:07:56.932271 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:07:56.932271 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:07:56.932335 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:07:56.932538 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:07:56.932539 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:07:56.932539 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:07:56.932685 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:07:56.932794 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0812 21:07:56.932891 (+ 97us) server_negotiation.cc:299] Negotiation successful
0812 21:07:56.932972 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":29}
I20260812 21:07:56.937026 16041 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 21:07:56.937758 16041 catalog_manager.cc:1504] Loading token signing keys...
I20260812 21:07:56.945775 16041 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 91178727cf734118870654aa0db441a9: Generated new TSK 0
I20260812 21:07:56.946038 16041 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 21:07:56.950030 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 15884
W20260812 21:07:56.959652 15857 connection.cc:535] client connection to 127.0.0.1:43711 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43711): Connection reset by peer (error 104)
2026-08-12T21:07:56Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1477 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786568607354921-12282-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 21:07:57 dist-test-slave-vpvm krb5kdc[16065](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 21:07:57 dist-test-slave-vpvm krb5kdc[16065](info): set up 2 sockets
Aug 12 21:07:57 dist-test-slave-vpvm krb5kdc[16065](info): commencing operation
krb5kdc: starting...
W20260812 21:07:58.140729 12282 mini_kdc.cc:121] Time spent starting KDC: real 1.167s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786568607354921-12282-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786568607354921-12282-0/krb5kdc/test-user.keytab.
Aug 12 21:07:58 dist-test-slave-vpvm krb5kdc[16065](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568878, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T21:07:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T21:07:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786568607354921-12282-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786568607354921-12282-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 21:07:58.268798 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786568607354921-12282-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786568607354921-12282-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786568607354921-12282-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39993
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786568607354921-12282-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38779
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39993
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786568607354921-12282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786568607354921-12282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786568607354921-12282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:07:58.352272 16094 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:07:58.352521 16094 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:07:58.352550 16094 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:07:58.353724 16094 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 21:07:58.353766 16094 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:07:58.353780 16094 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:07:58.353793 16094 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 21:07:58.353808 16094 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 21:07:58.355464 16094 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38779
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786568607354921-12282-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786568607354921-12282-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39993
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786568607354921-12282-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39993
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16094
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786568607354921-12282-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:07:58.355696 16094 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:07:58.355897 16094 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:07:58.361019 16104 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:07:58.361146 16094 server_base.cc:1034] running on GCE node
W20260812 21:07:58.361054 16103 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:07:58.361301 16113 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:07:58.361564 16094 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:07:58.361907 16094 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:07:58.363083 16094 hybrid_clock.cc:648] HybridClock initialized: now 1786568878363068 us; error 29 us; skew 500 ppm
Aug 12 21:07:58 dist-test-slave-vpvm krb5kdc[16065](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568878, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:07:58.365613 16094 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 21:07:58.366142 16094 webserver.cc:460] Webserver started at http://127.0.0.1:34803/ using document root <none> and password file <none>
I20260812 21:07:58.366379 16094 fs_manager.cc:359] Metadata directory not provided
I20260812 21:07:58.366429 16094 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:07:58.366568 16094 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:07:58.367682 16094 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786568607354921-12282-0/minicluster-data/master-0/data/instance:
uuid: "7ff2ee8c3d08409cb081158e0eed6c54"
format_stamp: "Formatted at 2026-08-12 21:07:58 on dist-test-slave-vpvm"
server_key: "e7899c58b8bcfcdc9fb67395a7f27638"
server_key_iv: "2cf2454c0f6458e73584b8ad62c83518"
server_key_version: "encryptionkey@0"
I20260812 21:07:58.368042 16094 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786568607354921-12282-0/minicluster-data/master-0/wal/instance:
uuid: "7ff2ee8c3d08409cb081158e0eed6c54"
format_stamp: "Formatted at 2026-08-12 21:07:58 on dist-test-slave-vpvm"
server_key: "e7899c58b8bcfcdc9fb67395a7f27638"
server_key_iv: "2cf2454c0f6458e73584b8ad62c83518"
server_key_version: "encryptionkey@0"
I20260812 21:07:58.369400 16094 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 21:07:58.370329 16128 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:07:58.370515 16094 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 21:07:58.370621 16094 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786568607354921-12282-0/minicluster-data/master-0/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786568607354921-12282-0/minicluster-data/master-0/wal
uuid: "7ff2ee8c3d08409cb081158e0eed6c54"
format_stamp: "Formatted at 2026-08-12 21:07:58 on dist-test-slave-vpvm"
server_key: "e7899c58b8bcfcdc9fb67395a7f27638"
server_key_iv: "2cf2454c0f6458e73584b8ad62c83518"
server_key_version: "encryptionkey@0"
I20260812 21:07:58.370697 16094 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786568607354921-12282-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786568607354921-12282-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786568607354921-12282-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:07:58.421000 16094 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:07:58.423352 16094 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:07:58.423514 16094 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:07:58.427557 16094 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39993
I20260812 21:07:58.427593 16221 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39993 every 8 connection(s)
I20260812 21:07:58.427971 16094 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
I20260812 21:07:58.428936 16222 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:07:58.431712 16222 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7ff2ee8c3d08409cb081158e0eed6c54: Bootstrap starting.
I20260812 21:07:58.432286 16222 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7ff2ee8c3d08409cb081158e0eed6c54: Neither blocks nor log segments found. Creating new log.
I20260812 21:07:58.432546 16222 log.cc:824] T 00000000000000000000000000000000 P 7ff2ee8c3d08409cb081158e0eed6c54: Log is configured to *not* fsync() on all Append() calls
I20260812 21:07:58.433271 16222 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7ff2ee8c3d08409cb081158e0eed6c54: No bootstrap required, opened a new log
I20260812 21:07:58.434919 16222 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7ff2ee8c3d08409cb081158e0eed6c54 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ff2ee8c3d08409cb081158e0eed6c54" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39993 } }
I20260812 21:07:58.435081 16222 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7ff2ee8c3d08409cb081158e0eed6c54 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:07:58.435128 16222 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7ff2ee8c3d08409cb081158e0eed6c54 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ff2ee8c3d08409cb081158e0eed6c54, State: Initialized, Role: FOLLOWER
I20260812 21:07:58.435253 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 16094
I20260812 21:07:58.435292 16222 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7ff2ee8c3d08409cb081158e0eed6c54 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ff2ee8c3d08409cb081158e0eed6c54" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39993 } }
I20260812 21:07:58.435364 12282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786568607354921-12282-0/minicluster-data/master-0/wal/instance
I20260812 21:07:58.435388 16222 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7ff2ee8c3d08409cb081158e0eed6c54 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 21:07:58.435420 16222 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7ff2ee8c3d08409cb081158e0eed6c54 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 21:07:58.435469 16222 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7ff2ee8c3d08409cb081158e0eed6c54 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:07:58.435567 12282 external_mini_cluster.cc:1428] Setting key cda3b6729296d6f6b59c59bf8dd85c12
I20260812 21:07:58.436144 16222 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7ff2ee8c3d08409cb081158e0eed6c54 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ff2ee8c3d08409cb081158e0eed6c54" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39993 } }
I20260812 21:07:58.436304 16222 leader_election.cc:302] T 00000000000000000000000000000000 P 7ff2ee8c3d08409cb081158e0eed6c54 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7ff2ee8c3d08409cb081158e0eed6c54; no voters:
I20260812 21:07:58.436476 16222 leader_election.cc:288] T 00000000000000000000000000000000 P 7ff2ee8c3d08409cb081158e0eed6c54 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 21:07:58.436507 16240 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7ff2ee8c3d08409cb081158e0eed6c54 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:07:58.436721 16240 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7ff2ee8c3d08409cb081158e0eed6c54 [term 1 LEADER]: Becoming Leader. State: Replica: 7ff2ee8c3d08409cb081158e0eed6c54, State: Running, Role: LEADER
I20260812 21:07:58.436753 16222 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7ff2ee8c3d08409cb081158e0eed6c54 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 21:07:58.436925 16240 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7ff2ee8c3d08409cb081158e0eed6c54 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ff2ee8c3d08409cb081158e0eed6c54" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39993 } }
I20260812 21:07:58.437356 16241 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ff2ee8c3d08409cb081158e0eed6c54 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7ff2ee8c3d08409cb081158e0eed6c54" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ff2ee8c3d08409cb081158e0eed6c54" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39993 } } }
I20260812 21:07:58.437453 16241 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ff2ee8c3d08409cb081158e0eed6c54 [sys.catalog]: This master's current role is: LEADER
I20260812 21:07:58.437655 16242 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ff2ee8c3d08409cb081158e0eed6c54 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7ff2ee8c3d08409cb081158e0eed6c54. Latest consensus state: current_term: 1 leader_uuid: "7ff2ee8c3d08409cb081158e0eed6c54" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ff2ee8c3d08409cb081158e0eed6c54" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39993 } } }
I20260812 21:07:58.437719 16242 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ff2ee8c3d08409cb081158e0eed6c54 [sys.catalog]: This master's current role is: LEADER
I20260812 21:07:58.438139 16258 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 21:07:58.438707 16258 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 21:07:58.440330 16258 catalog_manager.cc:1347] Generated new cluster ID: f753db27af0942bd8f7bf6f023f51b44
I20260812 21:07:58.440390 16258 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 21:07:58.442158 16232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:58.436089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.27:52487 (local address 127.0.0.1:39993)
0812 21:07:58.436280 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0812 21:07:58.436290 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 21:07:58.436325 (+ 35us) server_negotiation.cc:407] Connection header received
0812 21:07:58.436438 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:07:58.436446 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:07:58.436509 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:07:58.436663 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:07:58.437027 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:58.438862 (+ 1835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:58.440018 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:58.440594 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:58.440697 (+ 103us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:07:58.441410 (+ 713us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:07:58.441414 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:07:58.441415 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 21:07:58.441592 (+ 177us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:07:58.441630 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 21:07:58.441641 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:07:58.441712 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0812 21:07:58.441908 (+ 196us) server_negotiation.cc:299] Negotiation successful
0812 21:07:58.441937 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":87,"threads_started":1}
I20260812 21:07:58.443094 12282 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 21:07:58.447431 16232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:58.444452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.27:39305 (local address 127.0.0.1:39993)
0812 21:07:58.444498 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0812 21:07:58.444500 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:07:58.444557 (+ 57us) server_negotiation.cc:407] Connection header received
0812 21:07:58.444623 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:07:58.444624 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:07:58.444634 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:07:58.444678 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:07:58.444799 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:58.445519 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:58.446286 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:07:58.446511 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:07:58.446571 (+ 60us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:07:58.447200 (+ 629us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:07:58.447201 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:07:58.447201 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 21:07:58.447207 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:07:58.447224 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 21:07:58.447228 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:07:58.447273 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0812 21:07:58.447341 (+ 68us) server_negotiation.cc:299] Negotiation successful
0812 21:07:58.447354 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 21:07:58.452224 16258 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 21:07:58.452821 16258 catalog_manager.cc:1504] Loading token signing keys...
I20260812 21:07:58.455610 16258 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7ff2ee8c3d08409cb081158e0eed6c54: Generated new TSK 0
I20260812 21:07:58.455854 16258 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 21:07:58.458742 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 16094
W20260812 21:07:58.481168 16063 connection.cc:535] client connection to 127.0.0.1:39993 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39993): Connection reset by peer (error 104)
2026-08-12T21:07:58Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1521 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786568607354921-12282-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 21:07:58 dist-test-slave-vpvm krb5kdc[16283](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 21:07:58 dist-test-slave-vpvm krb5kdc[16283](info): set up 2 sockets
Aug 12 21:07:58 dist-test-slave-vpvm krb5kdc[16283](info): commencing operation
krb5kdc: starting...
W20260812 21:07:59.644088 12282 mini_kdc.cc:121] Time spent starting KDC: real 1.149s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786568607354921-12282-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786568607354921-12282-0/krb5kdc/test-user.keytab.
Aug 12 21:07:59 dist-test-slave-vpvm krb5kdc[16283](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568879, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T21:07:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T21:07:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786568607354921-12282-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786568607354921-12282-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 21:07:59.774214 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786568607354921-12282-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786568607354921-12282-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786568607354921-12282-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40591
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786568607354921-12282-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46653
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40591
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786568607354921-12282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786568607354921-12282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786568607354921-12282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:07:59.858392 16304 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:07:59.858626 16304 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:07:59.858659 16304 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:07:59.859822 16304 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 21:07:59.859866 16304 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:07:59.859885 16304 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:07:59.859902 16304 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 21:07:59.859917 16304 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 21:07:59.861389 16304 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46653
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786568607354921-12282-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786568607354921-12282-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40591
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786568607354921-12282-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40591
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16304
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786568607354921-12282-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:07:59.861644 16304 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:07:59.861853 16304 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:07:59.866907 16313 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:07:59.866876 16321 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:07:59.866873 16318 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:07:59.866912 16304 server_base.cc:1034] running on GCE node
I20260812 21:07:59.867398 16304 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:07:59.867748 16304 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:07:59.868908 16304 hybrid_clock.cc:648] HybridClock initialized: now 1786568879868890 us; error 40 us; skew 500 ppm
Aug 12 21:07:59 dist-test-slave-vpvm krb5kdc[16283](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568879, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:07:59.871577 16304 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 21:07:59.872053 16304 webserver.cc:460] Webserver started at http://127.0.0.1:45853/ using document root <none> and password file <none>
I20260812 21:07:59.872262 16304 fs_manager.cc:359] Metadata directory not provided
I20260812 21:07:59.872323 16304 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:07:59.872447 16304 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:07:59.873557 16304 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786568607354921-12282-0/minicluster-data/master-0/data/instance:
uuid: "e153dfba3df743bf91fd05cf2acd1925"
format_stamp: "Formatted at 2026-08-12 21:07:59 on dist-test-slave-vpvm"
server_key: "5c496a20346bcbec654ab813d8a9e094"
server_key_iv: "347b5d9b703932bac003ea94499c8add"
server_key_version: "encryptionkey@0"
I20260812 21:07:59.873898 16304 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786568607354921-12282-0/minicluster-data/master-0/wal/instance:
uuid: "e153dfba3df743bf91fd05cf2acd1925"
format_stamp: "Formatted at 2026-08-12 21:07:59 on dist-test-slave-vpvm"
server_key: "5c496a20346bcbec654ab813d8a9e094"
server_key_iv: "347b5d9b703932bac003ea94499c8add"
server_key_version: "encryptionkey@0"
I20260812 21:07:59.875233 16304 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 21:07:59.875981 16335 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:07:59.876173 16304 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 21:07:59.876259 16304 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786568607354921-12282-0/minicluster-data/master-0/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786568607354921-12282-0/minicluster-data/master-0/wal
uuid: "e153dfba3df743bf91fd05cf2acd1925"
format_stamp: "Formatted at 2026-08-12 21:07:59 on dist-test-slave-vpvm"
server_key: "5c496a20346bcbec654ab813d8a9e094"
server_key_iv: "347b5d9b703932bac003ea94499c8add"
server_key_version: "encryptionkey@0"
I20260812 21:07:59.876323 16304 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786568607354921-12282-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786568607354921-12282-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786568607354921-12282-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:07:59.879729 16304 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:07:59.881873 16304 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:07:59.882046 16304 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:07:59.886094 16304 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40591
I20260812 21:07:59.886137 16430 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40591 every 8 connection(s)
I20260812 21:07:59.886520 16304 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
I20260812 21:07:59.887540 16431 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:07:59.890012 16431 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e153dfba3df743bf91fd05cf2acd1925: Bootstrap starting.
I20260812 21:07:59.890187 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 16304
I20260812 21:07:59.890295 12282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786568607354921-12282-0/minicluster-data/master-0/wal/instance
I20260812 21:07:59.890493 12282 external_mini_cluster.cc:1428] Setting key 7663400a1e41e1c64f609239f283cabe
I20260812 21:07:59.890786 16431 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e153dfba3df743bf91fd05cf2acd1925: Neither blocks nor log segments found. Creating new log.
I20260812 21:07:59.891111 16431 log.cc:824] T 00000000000000000000000000000000 P e153dfba3df743bf91fd05cf2acd1925: Log is configured to *not* fsync() on all Append() calls
I20260812 21:07:59.891846 16431 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e153dfba3df743bf91fd05cf2acd1925: No bootstrap required, opened a new log
I20260812 21:07:59.893131 16437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:59.891030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.27:37105 (local address 127.0.0.1:40591)
0812 21:07:59.891254 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0812 21:07:59.891264 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 21:07:59.891299 (+ 35us) server_negotiation.cc:407] Connection header received
0812 21:07:59.891875 (+ 576us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:07:59.891884 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:07:59.891912 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:07:59.892059 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:07:59.892742 (+ 683us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:07:59.892745 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:07:59.892746 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 21:07:59.892750 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:07:59.892792 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 21:07:59.892802 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:07:59.892878 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 21:07:59.892911 (+ 33us) server_negotiation.cc:299] Negotiation successful
0812 21:07:59.892953 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":87,"threads_started":1}
I20260812 21:07:59.894124 12282 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 21:07:59.894176 16431 raft_consensus.cc:348] T 00000000000000000000000000000000 P e153dfba3df743bf91fd05cf2acd1925 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e153dfba3df743bf91fd05cf2acd1925" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40591 } }
I20260812 21:07:59.894338 16431 raft_consensus.cc:374] T 00000000000000000000000000000000 P e153dfba3df743bf91fd05cf2acd1925 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:07:59.894379 16431 raft_consensus.cc:729] T 00000000000000000000000000000000 P e153dfba3df743bf91fd05cf2acd1925 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e153dfba3df743bf91fd05cf2acd1925, State: Initialized, Role: FOLLOWER
I20260812 21:07:59.894505 16431 consensus_queue.cc:260] T 00000000000000000000000000000000 P e153dfba3df743bf91fd05cf2acd1925 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e153dfba3df743bf91fd05cf2acd1925" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40591 } }
I20260812 21:07:59.894579 16431 raft_consensus.cc:388] T 00000000000000000000000000000000 P e153dfba3df743bf91fd05cf2acd1925 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 21:07:59.894609 16431 raft_consensus.cc:482] T 00000000000000000000000000000000 P e153dfba3df743bf91fd05cf2acd1925 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 21:07:59.894656 16431 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e153dfba3df743bf91fd05cf2acd1925 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:07:59.895300 16431 raft_consensus.cc:504] T 00000000000000000000000000000000 P e153dfba3df743bf91fd05cf2acd1925 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e153dfba3df743bf91fd05cf2acd1925" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40591 } }
I20260812 21:07:59.895402 16431 leader_election.cc:302] T 00000000000000000000000000000000 P e153dfba3df743bf91fd05cf2acd1925 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e153dfba3df743bf91fd05cf2acd1925; no voters:
I20260812 21:07:59.895565 16431 leader_election.cc:288] T 00000000000000000000000000000000 P e153dfba3df743bf91fd05cf2acd1925 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 21:07:59.895583 16449 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e153dfba3df743bf91fd05cf2acd1925 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:07:59.895731 16449 raft_consensus.cc:686] T 00000000000000000000000000000000 P e153dfba3df743bf91fd05cf2acd1925 [term 1 LEADER]: Becoming Leader. State: Replica: e153dfba3df743bf91fd05cf2acd1925, State: Running, Role: LEADER
I20260812 21:07:59.895864 16431 sys_catalog.cc:564] T 00000000000000000000000000000000 P e153dfba3df743bf91fd05cf2acd1925 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 21:07:59.895884 16449 consensus_queue.cc:237] T 00000000000000000000000000000000 P e153dfba3df743bf91fd05cf2acd1925 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e153dfba3df743bf91fd05cf2acd1925" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40591 } }
I20260812 21:07:59.896584 16449 sys_catalog.cc:455] T 00000000000000000000000000000000 P e153dfba3df743bf91fd05cf2acd1925 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e153dfba3df743bf91fd05cf2acd1925" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e153dfba3df743bf91fd05cf2acd1925" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40591 } } }
I20260812 21:07:59.896701 16437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:07:59.895550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.27:54071 (local address 127.0.0.1:40591)
0812 21:07:59.895591 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0812 21:07:59.895593 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:07:59.895716 (+ 123us) server_negotiation.cc:407] Connection header received
0812 21:07:59.895773 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:07:59.895773 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:07:59.895778 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:07:59.895819 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:07:59.896414 (+ 595us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:07:59.896415 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:07:59.896415 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 21:07:59.896420 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:07:59.896435 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 21:07:59.896439 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:07:59.896469 (+ 30us) server_negotiation.cc:1035] Waiting for connection context
0812 21:07:59.896630 (+ 161us) server_negotiation.cc:299] Negotiation successful
0812 21:07:59.896646 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 21:07:59.896736 16449 sys_catalog.cc:458] T 00000000000000000000000000000000 P e153dfba3df743bf91fd05cf2acd1925 [sys.catalog]: This master's current role is: LEADER
I20260812 21:07:59.896869 16449 sys_catalog.cc:455] T 00000000000000000000000000000000 P e153dfba3df743bf91fd05cf2acd1925 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e153dfba3df743bf91fd05cf2acd1925. Latest consensus state: current_term: 1 leader_uuid: "e153dfba3df743bf91fd05cf2acd1925" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e153dfba3df743bf91fd05cf2acd1925" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40591 } } }
I20260812 21:07:59.897017 16449 sys_catalog.cc:458] T 00000000000000000000000000000000 P e153dfba3df743bf91fd05cf2acd1925 [sys.catalog]: This master's current role is: LEADER
W20260812 21:07:59.897615 16479 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e153dfba3df743bf91fd05cf2acd1925: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 21:07:59.897675 16479 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 21:07:59.897723 16468 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 21:07:59.897892 16468 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 21:07:59.899266 16468 catalog_manager.cc:1347] Generated new cluster ID: 72dc6a2fc7df483684852028a27fe5c1
I20260812 21:07:59.899317 16468 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 21:07:59.915323 16468 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 21:07:59.915549 16468 catalog_manager.cc:1504] Loading token signing keys...
I20260812 21:07:59.926486 16468 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e153dfba3df743bf91fd05cf2acd1925: Generated new TSK 0
I20260812 21:07:59.926749 16468 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 21:07:59.936600 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 16304
2026-08-12T21:07:59Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1491 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786568607354921-12282-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 21:08:00 dist-test-slave-vpvm krb5kdc[16495](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 21:08:00 dist-test-slave-vpvm krb5kdc[16495](info): set up 2 sockets
Aug 12 21:08:00 dist-test-slave-vpvm krb5kdc[16495](info): commencing operation
krb5kdc: starting...
W20260812 21:08:01.147837 12282 mini_kdc.cc:121] Time spent starting KDC: real 1.162s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786568607354921-12282-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786568607354921-12282-0/krb5kdc/test-user.keytab.
Aug 12 21:08:01 dist-test-slave-vpvm krb5kdc[16495](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568881, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T21:08:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T21:08:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.254.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.254.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.254.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.190.keytab.
Entry for principal kudu/127.11.254.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.190.keytab.
I20260812 21:08:01.273609 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786568607354921-12282-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786568607354921-12282-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786568607354921-12282-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.254.190:42585
--webserver_interface=127.11.254.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.190.keytab
--principal=kudu/127.11.254.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.254.148:36463
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.254.190:42585
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786568607354921-12282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786568607354921-12282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786568607354921-12282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:08:01.357218 16518 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:08:01.357446 16518 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:08:01.357479 16518 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:08:01.358700 16518 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 21:08:01.358744 16518 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:08:01.358762 16518 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:08:01.358778 16518 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 21:08:01.358793 16518 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 21:08:01.360329 16518 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.254.148:36463
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786568607354921-12282-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786568607354921-12282-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.254.190:42585
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.190.keytab
--principal=kudu/127.11.254.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.254.190:42585
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.254.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16518
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786568607354921-12282-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:08:01.360599 16518 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:08:01.360816 16518 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:08:01.365674 16529 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:08:01.365703 16527 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:08:01.365703 16533 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:08:01.365978 16518 server_base.cc:1034] running on GCE node
I20260812 21:08:01.366209 16518 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:08:01.366494 16518 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:08:01.367660 16518 hybrid_clock.cc:648] HybridClock initialized: now 1786568881367641 us; error 34 us; skew 500 ppm
Aug 12 21:08:01 dist-test-slave-vpvm krb5kdc[16495](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568881, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:08:01.377014 16518 init.cc:377] Logged in from keytab as kudu/127.11.254.190@KRBTEST.COM (short username kudu)
I20260812 21:08:01.377537 16518 webserver.cc:460] Webserver started at http://127.11.254.190:36951/ using document root <none> and password file <none>
I20260812 21:08:01.377780 16518 fs_manager.cc:359] Metadata directory not provided
I20260812 21:08:01.377832 16518 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:08:01.378028 16518 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:08:01.379225 16518 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786568607354921-12282-0/minicluster-data/master-0/data/instance:
uuid: "1547fbe1b64a414699692bd97ca9fffa"
format_stamp: "Formatted at 2026-08-12 21:08:01 on dist-test-slave-vpvm"
server_key: "b4bc8b6fcb6ead2204e5fc7e691455ce"
server_key_iv: "686c3d1cdbe67fe0fa310a7d1f15e8f6"
server_key_version: "encryptionkey@0"
I20260812 21:08:01.379586 16518 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786568607354921-12282-0/minicluster-data/master-0/wal/instance:
uuid: "1547fbe1b64a414699692bd97ca9fffa"
format_stamp: "Formatted at 2026-08-12 21:08:01 on dist-test-slave-vpvm"
server_key: "b4bc8b6fcb6ead2204e5fc7e691455ce"
server_key_iv: "686c3d1cdbe67fe0fa310a7d1f15e8f6"
server_key_version: "encryptionkey@0"
I20260812 21:08:01.380990 16518 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 21:08:01.381870 16546 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:08:01.382099 16518 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 21:08:01.382201 16518 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786568607354921-12282-0/minicluster-data/master-0/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786568607354921-12282-0/minicluster-data/master-0/wal
uuid: "1547fbe1b64a414699692bd97ca9fffa"
format_stamp: "Formatted at 2026-08-12 21:08:01 on dist-test-slave-vpvm"
server_key: "b4bc8b6fcb6ead2204e5fc7e691455ce"
server_key_iv: "686c3d1cdbe67fe0fa310a7d1f15e8f6"
server_key_version: "encryptionkey@0"
I20260812 21:08:01.382273 16518 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786568607354921-12282-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786568607354921-12282-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786568607354921-12282-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:08:01.400955 16518 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:08:01.403224 16518 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:08:01.403393 16518 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:08:01.407829 16518 rpc_server.cc:307] RPC server started. Bound to: 127.11.254.190:42585
I20260812 21:08:01.407867 16649 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.254.190:42585 every 8 connection(s)
I20260812 21:08:01.408257 16518 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
I20260812 21:08:01.409145 16651 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:08:01.409569 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 16518
I20260812 21:08:01.409658 12282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786568607354921-12282-0/minicluster-data/master-0/wal/instance
I20260812 21:08:01.409849 12282 external_mini_cluster.cc:1428] Setting key 9e96a145e14487082ecfd654433e7fe4
I20260812 21:08:01.412139 16651 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1547fbe1b64a414699692bd97ca9fffa: Bootstrap starting.
I20260812 21:08:01.412878 16651 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1547fbe1b64a414699692bd97ca9fffa: Neither blocks nor log segments found. Creating new log.
I20260812 21:08:01.413208 16651 log.cc:824] T 00000000000000000000000000000000 P 1547fbe1b64a414699692bd97ca9fffa: Log is configured to *not* fsync() on all Append() calls
I20260812 21:08:01.414007 16651 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1547fbe1b64a414699692bd97ca9fffa: No bootstrap required, opened a new log
I20260812 21:08:01.416278 16651 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1547fbe1b64a414699692bd97ca9fffa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1547fbe1b64a414699692bd97ca9fffa" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 42585 } }
I20260812 21:08:01.416424 16651 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1547fbe1b64a414699692bd97ca9fffa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:08:01.416446 16651 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1547fbe1b64a414699692bd97ca9fffa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1547fbe1b64a414699692bd97ca9fffa, State: Initialized, Role: FOLLOWER
I20260812 21:08:01.416571 16651 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1547fbe1b64a414699692bd97ca9fffa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1547fbe1b64a414699692bd97ca9fffa" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 42585 } }
I20260812 21:08:01.416644 16651 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1547fbe1b64a414699692bd97ca9fffa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 21:08:01.416668 16651 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1547fbe1b64a414699692bd97ca9fffa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 21:08:01.416715 16651 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1547fbe1b64a414699692bd97ca9fffa [term 0 FOLLOWER]: Advancing to term 1
Aug 12 21:08:01 dist-test-slave-vpvm krb5kdc[16495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568881, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.254.190@KRBTEST.COM
I20260812 21:08:01.417299 16651 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1547fbe1b64a414699692bd97ca9fffa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1547fbe1b64a414699692bd97ca9fffa" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 42585 } }
I20260812 21:08:01.417392 16651 leader_election.cc:302] T 00000000000000000000000000000000 P 1547fbe1b64a414699692bd97ca9fffa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1547fbe1b64a414699692bd97ca9fffa; no voters:
I20260812 21:08:01.417567 16651 leader_election.cc:288] T 00000000000000000000000000000000 P 1547fbe1b64a414699692bd97ca9fffa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 21:08:01.417616 16658 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1547fbe1b64a414699692bd97ca9fffa [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:08:01.417783 16658 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1547fbe1b64a414699692bd97ca9fffa [term 1 LEADER]: Becoming Leader. State: Replica: 1547fbe1b64a414699692bd97ca9fffa, State: Running, Role: LEADER
I20260812 21:08:01.417858 16651 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1547fbe1b64a414699692bd97ca9fffa [sys.catalog]: configured and running, proceeding with master startup.
I20260812 21:08:01.418009 16658 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1547fbe1b64a414699692bd97ca9fffa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1547fbe1b64a414699692bd97ca9fffa" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 42585 } }
I20260812 21:08:01.418715 16661 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1547fbe1b64a414699692bd97ca9fffa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1547fbe1b64a414699692bd97ca9fffa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1547fbe1b64a414699692bd97ca9fffa" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 42585 } } }
I20260812 21:08:01.418812 16661 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1547fbe1b64a414699692bd97ca9fffa [sys.catalog]: This master's current role is: LEADER
I20260812 21:08:01.418934 16658 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1547fbe1b64a414699692bd97ca9fffa [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1547fbe1b64a414699692bd97ca9fffa. Latest consensus state: current_term: 1 leader_uuid: "1547fbe1b64a414699692bd97ca9fffa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1547fbe1b64a414699692bd97ca9fffa" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 42585 } } }
I20260812 21:08:01.419104 16658 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1547fbe1b64a414699692bd97ca9fffa [sys.catalog]: This master's current role is: LEADER
W20260812 21:08:01.419786 16686 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1547fbe1b64a414699692bd97ca9fffa: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 21:08:01.419833 16686 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 21:08:01.419929 16678 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 21:08:01.420104 16678 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 21:08:01.421537 16678 catalog_manager.cc:1347] Generated new cluster ID: c5de3ec1db9e49b3a46e70eabedd2240
I20260812 21:08:01.421584 16678 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 21:08:01.422003 16656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:01.410442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44321 (local address 127.11.254.190:42585)
0812 21:08:01.410708 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:01.410719 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:01.410758 (+ 39us) server_negotiation.cc:407] Connection header received
0812 21:08:01.411317 (+ 559us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:01.411327 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:01.411390 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:01.411550 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:01.412206 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:01.413537 (+ 1331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:01.414396 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:01.415007 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:01.415124 (+ 117us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:01.417742 (+ 2618us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:08:01.417748 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:08:01.417748 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:08:01.417762 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:08:01.419910 (+ 2148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:01.420466 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:01.420471 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:01.420471 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:01.420589 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:01.420894 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:01.420895 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:01.420895 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:01.421214 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:08:01.421350 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:01.421668 (+ 318us) server_negotiation.cc:299] Negotiation successful
0812 21:08:01.421783 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":210,"thread_start_us":151,"threads_started":1}
I20260812 21:08:01.427191 12282 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 21:08:01.437666 16656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:01.428845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39315 (local address 127.11.254.190:42585)
0812 21:08:01.428893 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:01.428895 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:01.430105 (+ 1210us) server_negotiation.cc:407] Connection header received
0812 21:08:01.430213 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:01.430214 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:01.430229 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:01.430307 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:01.430950 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:01.431731 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:01.432739 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:01.433091 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:01.433163 (+ 72us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:01.434666 (+ 1503us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:08:01.434667 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:08:01.434668 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:08:01.434676 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:08:01.436436 (+ 1760us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:01.436871 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:01.436872 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:01.436872 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:01.436942 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:01.437132 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:01.437133 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:01.437133 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:01.437277 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:08:01.437405 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:01.437488 (+ 83us) server_negotiation.cc:299] Negotiation successful
0812 21:08:01.437559 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 21:08:01.443364 16678 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 21:08:01.443914 16678 catalog_manager.cc:1504] Loading token signing keys...
I20260812 21:08:01.447377 16678 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1547fbe1b64a414699692bd97ca9fffa: Generated new TSK 0
I20260812 21:08:01.447590 16678 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 21:08:01.453075 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 16518
W20260812 21:08:01.465042 16492 connection.cc:535] client connection to 127.11.254.190:42585 recv error: Network error: failed to read from TLS socket (remote: 127.11.254.190:42585): Connection reset by peer (error 104)
2026-08-12T21:08:01Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1492 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786568607354921-12282-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 21:08:01 dist-test-slave-vpvm krb5kdc[16709](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 21:08:01 dist-test-slave-vpvm krb5kdc[16709](info): set up 2 sockets
Aug 12 21:08:01 dist-test-slave-vpvm krb5kdc[16709](info): commencing operation
krb5kdc: starting...
W20260812 21:08:02.641355 12282 mini_kdc.cc:121] Time spent starting KDC: real 1.163s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786568607354921-12282-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786568607354921-12282-0/krb5kdc/test-user.keytab.
Aug 12 21:08:02 dist-test-slave-vpvm krb5kdc[16709](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568882, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T21:08:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T21:08:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.254.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.254.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.254.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.190.keytab.
Entry for principal kudu/127.11.254.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.190.keytab.
I20260812 21:08:02.769418 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786568607354921-12282-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786568607354921-12282-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786568607354921-12282-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.254.190:38997
--webserver_interface=127.11.254.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.190.keytab
--principal=kudu/127.11.254.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.254.148:45595
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.254.190:38997
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786568607354921-12282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786568607354921-12282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786568607354921-12282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:08:02.853976 16733 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:08:02.854189 16733 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:08:02.854212 16733 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:08:02.855325 16733 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 21:08:02.855360 16733 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:08:02.855373 16733 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:08:02.855384 16733 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 21:08:02.855396 16733 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 21:08:02.856825 16733 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.254.148:45595
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786568607354921-12282-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786568607354921-12282-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.254.190:38997
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.190.keytab
--principal=kudu/127.11.254.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.254.190:38997
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.254.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16733
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786568607354921-12282-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:08:02.857016 16733 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:08:02.857203 16733 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:08:02.862118 16745 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:08:02.862119 16749 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:08:02.862265 16733 server_base.cc:1034] running on GCE node
W20260812 21:08:02.862082 16752 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:08:02.862548 16733 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:08:02.862826 16733 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:08:02.863978 16733 hybrid_clock.cc:648] HybridClock initialized: now 1786568882863956 us; error 33 us; skew 500 ppm
Aug 12 21:08:02 dist-test-slave-vpvm krb5kdc[16709](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568882, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:08:02.866494 16733 init.cc:377] Logged in from keytab as kudu/127.11.254.190@KRBTEST.COM (short username kudu)
I20260812 21:08:02.866955 16733 webserver.cc:460] Webserver started at http://127.11.254.190:33603/ using document root <none> and password file <none>
I20260812 21:08:02.867194 16733 fs_manager.cc:359] Metadata directory not provided
I20260812 21:08:02.867244 16733 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:08:02.867369 16733 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:08:02.868455 16733 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786568607354921-12282-0/minicluster-data/master-0/data/instance:
uuid: "487eb30389604e738d25ac043d51e8a2"
format_stamp: "Formatted at 2026-08-12 21:08:02 on dist-test-slave-vpvm"
server_key: "587f513f09fabacd1e3c7be7ff529e70"
server_key_iv: "a685da5a81bd5d69c5da9259a62a8195"
server_key_version: "encryptionkey@0"
I20260812 21:08:02.868798 16733 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786568607354921-12282-0/minicluster-data/master-0/wal/instance:
uuid: "487eb30389604e738d25ac043d51e8a2"
format_stamp: "Formatted at 2026-08-12 21:08:02 on dist-test-slave-vpvm"
server_key: "587f513f09fabacd1e3c7be7ff529e70"
server_key_iv: "a685da5a81bd5d69c5da9259a62a8195"
server_key_version: "encryptionkey@0"
I20260812 21:08:02.870142 16733 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 21:08:02.870857 16761 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:08:02.871003 16733 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 21:08:02.871102 16733 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786568607354921-12282-0/minicluster-data/master-0/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786568607354921-12282-0/minicluster-data/master-0/wal
uuid: "487eb30389604e738d25ac043d51e8a2"
format_stamp: "Formatted at 2026-08-12 21:08:02 on dist-test-slave-vpvm"
server_key: "587f513f09fabacd1e3c7be7ff529e70"
server_key_iv: "a685da5a81bd5d69c5da9259a62a8195"
server_key_version: "encryptionkey@0"
I20260812 21:08:02.871166 16733 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786568607354921-12282-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786568607354921-12282-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786568607354921-12282-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:08:02.885669 16733 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:08:02.887925 16733 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:08:02.888089 16733 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:08:02.892237 16733 rpc_server.cc:307] RPC server started. Bound to: 127.11.254.190:38997
I20260812 21:08:02.892277 16853 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.254.190:38997 every 8 connection(s)
I20260812 21:08:02.892624 16733 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
I20260812 21:08:02.893527 16854 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:08:02.895346 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 16733
I20260812 21:08:02.895440 12282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786568607354921-12282-0/minicluster-data/master-0/wal/instance
I20260812 21:08:02.895622 12282 external_mini_cluster.cc:1428] Setting key 72557b1523d090e7341651cdd578b45a
I20260812 21:08:02.896150 16854 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 487eb30389604e738d25ac043d51e8a2: Bootstrap starting.
I20260812 21:08:02.896752 16854 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 487eb30389604e738d25ac043d51e8a2: Neither blocks nor log segments found. Creating new log.
I20260812 21:08:02.897032 16854 log.cc:824] T 00000000000000000000000000000000 P 487eb30389604e738d25ac043d51e8a2: Log is configured to *not* fsync() on all Append() calls
I20260812 21:08:02.897665 16854 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 487eb30389604e738d25ac043d51e8a2: No bootstrap required, opened a new log
I20260812 21:08:02.899492 16854 raft_consensus.cc:348] T 00000000000000000000000000000000 P 487eb30389604e738d25ac043d51e8a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "487eb30389604e738d25ac043d51e8a2" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 38997 } }
I20260812 21:08:02.899641 16854 raft_consensus.cc:374] T 00000000000000000000000000000000 P 487eb30389604e738d25ac043d51e8a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:08:02.899673 16854 raft_consensus.cc:729] T 00000000000000000000000000000000 P 487eb30389604e738d25ac043d51e8a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 487eb30389604e738d25ac043d51e8a2, State: Initialized, Role: FOLLOWER
I20260812 21:08:02.899824 16854 consensus_queue.cc:260] T 00000000000000000000000000000000 P 487eb30389604e738d25ac043d51e8a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "487eb30389604e738d25ac043d51e8a2" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 38997 } }
I20260812 21:08:02.899895 16854 raft_consensus.cc:388] T 00000000000000000000000000000000 P 487eb30389604e738d25ac043d51e8a2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 21:08:02.899924 16854 raft_consensus.cc:482] T 00000000000000000000000000000000 P 487eb30389604e738d25ac043d51e8a2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 21:08:02.899972 16854 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 487eb30389604e738d25ac043d51e8a2 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:08:02.900507 16854 raft_consensus.cc:504] T 00000000000000000000000000000000 P 487eb30389604e738d25ac043d51e8a2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "487eb30389604e738d25ac043d51e8a2" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 38997 } }
I20260812 21:08:02.900604 16854 leader_election.cc:302] T 00000000000000000000000000000000 P 487eb30389604e738d25ac043d51e8a2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 487eb30389604e738d25ac043d51e8a2; no voters:
I20260812 21:08:02.900766 16854 leader_election.cc:288] T 00000000000000000000000000000000 P 487eb30389604e738d25ac043d51e8a2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 21:08:02.900795 16858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:02.896143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57853 (local address 127.11.254.190:38997)
0812 21:08:02.896327 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:02.896337 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:02.896367 (+ 30us) server_negotiation.cc:407] Connection header received
0812 21:08:02.896838 (+ 471us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:02.896847 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:02.896912 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:02.897061 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:02.897273 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:02.898295 (+ 1022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:02.899043 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:02.899460 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:02.899555 (+ 95us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:02.900200 (+ 645us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:08:02.900204 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:08:02.900204 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 21:08:02.900369 (+ 165us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:08:02.900403 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 21:08:02.900413 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:08:02.900481 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:02.900570 (+ 89us) server_negotiation.cc:299] Negotiation successful
0812 21:08:02.900607 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":75,"threads_started":1}
I20260812 21:08:02.900808 16860 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 487eb30389604e738d25ac043d51e8a2 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:08:02.900954 16860 raft_consensus.cc:686] T 00000000000000000000000000000000 P 487eb30389604e738d25ac043d51e8a2 [term 1 LEADER]: Becoming Leader. State: Replica: 487eb30389604e738d25ac043d51e8a2, State: Running, Role: LEADER
I20260812 21:08:02.901012 16854 sys_catalog.cc:564] T 00000000000000000000000000000000 P 487eb30389604e738d25ac043d51e8a2 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 21:08:02.901034 16860 consensus_queue.cc:237] T 00000000000000000000000000000000 P 487eb30389604e738d25ac043d51e8a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "487eb30389604e738d25ac043d51e8a2" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 38997 } }
I20260812 21:08:02.901458 16867 sys_catalog.cc:455] T 00000000000000000000000000000000 P 487eb30389604e738d25ac043d51e8a2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "487eb30389604e738d25ac043d51e8a2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "487eb30389604e738d25ac043d51e8a2" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 38997 } } }
I20260812 21:08:02.901592 16867 sys_catalog.cc:458] T 00000000000000000000000000000000 P 487eb30389604e738d25ac043d51e8a2 [sys.catalog]: This master's current role is: LEADER
I20260812 21:08:02.901717 12282 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 21:08:02.902341 16879 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 21:08:02.901542 16868 sys_catalog.cc:455] T 00000000000000000000000000000000 P 487eb30389604e738d25ac043d51e8a2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 487eb30389604e738d25ac043d51e8a2. Latest consensus state: current_term: 1 leader_uuid: "487eb30389604e738d25ac043d51e8a2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "487eb30389604e738d25ac043d51e8a2" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 38997 } } }
I20260812 21:08:02.902626 16868 sys_catalog.cc:458] T 00000000000000000000000000000000 P 487eb30389604e738d25ac043d51e8a2 [sys.catalog]: This master's current role is: LEADER
I20260812 21:08:02.902944 16879 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 21:08:02.904515 16879 catalog_manager.cc:1347] Generated new cluster ID: 354447bf6b714a48b5b316df53be671d
I20260812 21:08:02.904568 16879 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 21:08:02.906494 16858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:02.903255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36907 (local address 127.11.254.190:38997)
0812 21:08:02.903288 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:02.903290 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:02.903399 (+ 109us) server_negotiation.cc:407] Connection header received
0812 21:08:02.903448 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:02.903449 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:02.903459 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:02.903522 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:02.903638 (+ 116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:02.904227 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:02.905175 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:02.905466 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:02.905498 (+ 32us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:02.906262 (+ 764us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:08:02.906264 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:08:02.906264 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 21:08:02.906268 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:08:02.906281 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 21:08:02.906284 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:08:02.906312 (+ 28us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:02.906411 (+ 99us) server_negotiation.cc:299] Negotiation successful
0812 21:08:02.906422 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260812 21:08:02.923245 16879 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 21:08:02.923813 16879 catalog_manager.cc:1504] Loading token signing keys...
I20260812 21:08:02.935037 16879 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 487eb30389604e738d25ac043d51e8a2: Generated new TSK 0
I20260812 21:08:02.935258 16879 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 21:08:02.937506 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 16733
W20260812 21:08:02.953714 16705 connection.cc:535] client connection to 127.11.254.190:38997 recv error: Network error: failed to read from TLS socket (remote: 127.11.254.190:38997): Connection reset by peer (error 104)
2026-08-12T21:08:02Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1488 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786568607354921-12282-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 21:08:03 dist-test-slave-vpvm krb5kdc[16911](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 21:08:03 dist-test-slave-vpvm krb5kdc[16911](info): set up 2 sockets
Aug 12 21:08:03 dist-test-slave-vpvm krb5kdc[16911](info): commencing operation
krb5kdc: starting...
W20260812 21:08:04.122752 12282 mini_kdc.cc:121] Time spent starting KDC: real 1.155s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786568607354921-12282-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786568607354921-12282-0/krb5kdc/test-user.keytab.
Aug 12 21:08:04 dist-test-slave-vpvm krb5kdc[16911](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568884, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T21:08:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T21:08:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.254.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.254.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.254.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.190.keytab.
Entry for principal kudu/127.11.254.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.190.keytab.
I20260812 21:08:04.249431 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786568607354921-12282-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786568607354921-12282-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786568607354921-12282-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.254.190:40747
--webserver_interface=127.11.254.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.190.keytab
--principal=kudu/127.11.254.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.254.148:42539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.254.190:40747
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786568607354921-12282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786568607354921-12282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786568607354921-12282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:08:04.335909 16938 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:08:04.336127 16938 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:08:04.336149 16938 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:08:04.337301 16938 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 21:08:04.337347 16938 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:08:04.337360 16938 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:08:04.337371 16938 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 21:08:04.337383 16938 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 21:08:04.338949 16938 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.254.148:42539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786568607354921-12282-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786568607354921-12282-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.254.190:40747
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.190.keytab
--principal=kudu/127.11.254.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.254.190:40747
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.254.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16938
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786568607354921-12282-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:08:04.339195 16938 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:08:04.339424 16938 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:08:04.344455 16953 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:08:04.344512 16949 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:08:04.344503 16948 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:08:04.344580 16938 server_base.cc:1034] running on GCE node
I20260812 21:08:04.345013 16938 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:08:04.345329 16938 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:08:04.346511 16938 hybrid_clock.cc:648] HybridClock initialized: now 1786568884346487 us; error 36 us; skew 500 ppm
Aug 12 21:08:04 dist-test-slave-vpvm krb5kdc[16911](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568884, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:08:04.349213 16938 init.cc:377] Logged in from keytab as kudu/127.11.254.190@KRBTEST.COM (short username kudu)
I20260812 21:08:04.349658 16938 webserver.cc:460] Webserver started at http://127.11.254.190:42011/ using document root <none> and password file <none>
I20260812 21:08:04.349869 16938 fs_manager.cc:359] Metadata directory not provided
I20260812 21:08:04.349918 16938 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:08:04.350090 16938 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:08:04.351192 16938 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786568607354921-12282-0/minicluster-data/master-0/data/instance:
uuid: "45ecf321ffe949599e93c8db69d6f0a1"
format_stamp: "Formatted at 2026-08-12 21:08:04 on dist-test-slave-vpvm"
server_key: "9d7d491abd00206f9e8aa7d523529822"
server_key_iv: "5342d6b68fdbc3c198d08aa0baf2f30e"
server_key_version: "encryptionkey@0"
I20260812 21:08:04.351504 16938 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786568607354921-12282-0/minicluster-data/master-0/wal/instance:
uuid: "45ecf321ffe949599e93c8db69d6f0a1"
format_stamp: "Formatted at 2026-08-12 21:08:04 on dist-test-slave-vpvm"
server_key: "9d7d491abd00206f9e8aa7d523529822"
server_key_iv: "5342d6b68fdbc3c198d08aa0baf2f30e"
server_key_version: "encryptionkey@0"
I20260812 21:08:04.352815 16938 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 21:08:04.353510 16967 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:08:04.353660 16938 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 21:08:04.353749 16938 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786568607354921-12282-0/minicluster-data/master-0/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786568607354921-12282-0/minicluster-data/master-0/wal
uuid: "45ecf321ffe949599e93c8db69d6f0a1"
format_stamp: "Formatted at 2026-08-12 21:08:04 on dist-test-slave-vpvm"
server_key: "9d7d491abd00206f9e8aa7d523529822"
server_key_iv: "5342d6b68fdbc3c198d08aa0baf2f30e"
server_key_version: "encryptionkey@0"
I20260812 21:08:04.353811 16938 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786568607354921-12282-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786568607354921-12282-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786568607354921-12282-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:08:04.357669 16938 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:08:04.359532 16938 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:08:04.359674 16938 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:08:04.363767 16938 rpc_server.cc:307] RPC server started. Bound to: 127.11.254.190:40747
I20260812 21:08:04.363801 17075 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.254.190:40747 every 8 connection(s)
I20260812 21:08:04.364135 16938 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
I20260812 21:08:04.365161 17077 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:08:04.365736 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 16938
I20260812 21:08:04.365840 12282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786568607354921-12282-0/minicluster-data/master-0/wal/instance
I20260812 21:08:04.366073 12282 external_mini_cluster.cc:1428] Setting key b7576330972a0a45b4a08dff0978b208
I20260812 21:08:04.368337 17077 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 45ecf321ffe949599e93c8db69d6f0a1: Bootstrap starting.
I20260812 21:08:04.368573 17085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:04.366593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48099 (local address 127.11.254.190:40747)
0812 21:08:04.366786 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:04.366797 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:04.366835 (+ 38us) server_negotiation.cc:407] Connection header received
0812 21:08:04.367366 (+ 531us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:04.367375 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:04.367411 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:04.367568 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:04.368145 (+ 577us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:08:04.368149 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:08:04.368150 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 21:08:04.368154 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:08:04.368199 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 21:08:04.368213 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:08:04.368281 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:04.368338 (+ 57us) server_negotiation.cc:299] Negotiation successful
0812 21:08:04.368387 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":81,"threads_started":1}
I20260812 21:08:04.369083 17077 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 45ecf321ffe949599e93c8db69d6f0a1: Neither blocks nor log segments found. Creating new log.
I20260812 21:08:04.369388 17077 log.cc:824] T 00000000000000000000000000000000 P 45ecf321ffe949599e93c8db69d6f0a1: Log is configured to *not* fsync() on all Append() calls
I20260812 21:08:04.369601 12282 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 21:08:04.370126 17077 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 45ecf321ffe949599e93c8db69d6f0a1: No bootstrap required, opened a new log
I20260812 21:08:04.371896 17085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:04.370960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44363 (local address 127.11.254.190:40747)
0812 21:08:04.370999 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:04.371001 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:04.371066 (+ 65us) server_negotiation.cc:407] Connection header received
0812 21:08:04.371107 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:04.371108 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:04.371113 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:04.371178 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:04.371692 (+ 514us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:08:04.371693 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:08:04.371693 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 21:08:04.371698 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:08:04.371714 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 21:08:04.371719 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:08:04.371786 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:04.371819 (+ 33us) server_negotiation.cc:299] Negotiation successful
0812 21:08:04.371830 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 21:08:04.372273 17077 raft_consensus.cc:348] T 00000000000000000000000000000000 P 45ecf321ffe949599e93c8db69d6f0a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45ecf321ffe949599e93c8db69d6f0a1" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 40747 } }
I20260812 21:08:04.372413 17077 raft_consensus.cc:374] T 00000000000000000000000000000000 P 45ecf321ffe949599e93c8db69d6f0a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:08:04.372444 17077 raft_consensus.cc:729] T 00000000000000000000000000000000 P 45ecf321ffe949599e93c8db69d6f0a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45ecf321ffe949599e93c8db69d6f0a1, State: Initialized, Role: FOLLOWER
I20260812 21:08:04.372560 17077 consensus_queue.cc:260] T 00000000000000000000000000000000 P 45ecf321ffe949599e93c8db69d6f0a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45ecf321ffe949599e93c8db69d6f0a1" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 40747 } }
I20260812 21:08:04.372651 17077 raft_consensus.cc:388] T 00000000000000000000000000000000 P 45ecf321ffe949599e93c8db69d6f0a1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 21:08:04.372684 17077 raft_consensus.cc:482] T 00000000000000000000000000000000 P 45ecf321ffe949599e93c8db69d6f0a1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 21:08:04.372731 17077 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 45ecf321ffe949599e93c8db69d6f0a1 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:08:04.373267 17077 raft_consensus.cc:504] T 00000000000000000000000000000000 P 45ecf321ffe949599e93c8db69d6f0a1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45ecf321ffe949599e93c8db69d6f0a1" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 40747 } }
I20260812 21:08:04.373365 17077 leader_election.cc:302] T 00000000000000000000000000000000 P 45ecf321ffe949599e93c8db69d6f0a1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 45ecf321ffe949599e93c8db69d6f0a1; no voters:
I20260812 21:08:04.373515 17077 leader_election.cc:288] T 00000000000000000000000000000000 P 45ecf321ffe949599e93c8db69d6f0a1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 21:08:04.373595 17093 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 45ecf321ffe949599e93c8db69d6f0a1 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:08:04.373739 17093 raft_consensus.cc:686] T 00000000000000000000000000000000 P 45ecf321ffe949599e93c8db69d6f0a1 [term 1 LEADER]: Becoming Leader. State: Replica: 45ecf321ffe949599e93c8db69d6f0a1, State: Running, Role: LEADER
I20260812 21:08:04.373771 17077 sys_catalog.cc:564] T 00000000000000000000000000000000 P 45ecf321ffe949599e93c8db69d6f0a1 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 21:08:04.373838 17093 consensus_queue.cc:237] T 00000000000000000000000000000000 P 45ecf321ffe949599e93c8db69d6f0a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45ecf321ffe949599e93c8db69d6f0a1" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 40747 } }
I20260812 21:08:04.374224 17094 sys_catalog.cc:455] T 00000000000000000000000000000000 P 45ecf321ffe949599e93c8db69d6f0a1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "45ecf321ffe949599e93c8db69d6f0a1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45ecf321ffe949599e93c8db69d6f0a1" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 40747 } } }
I20260812 21:08:04.374231 17095 sys_catalog.cc:455] T 00000000000000000000000000000000 P 45ecf321ffe949599e93c8db69d6f0a1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 45ecf321ffe949599e93c8db69d6f0a1. Latest consensus state: current_term: 1 leader_uuid: "45ecf321ffe949599e93c8db69d6f0a1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45ecf321ffe949599e93c8db69d6f0a1" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 40747 } } }
I20260812 21:08:04.374339 17094 sys_catalog.cc:458] T 00000000000000000000000000000000 P 45ecf321ffe949599e93c8db69d6f0a1 [sys.catalog]: This master's current role is: LEADER
I20260812 21:08:04.374346 17095 sys_catalog.cc:458] T 00000000000000000000000000000000 P 45ecf321ffe949599e93c8db69d6f0a1 [sys.catalog]: This master's current role is: LEADER
W20260812 21:08:04.375530 17125 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 45ecf321ffe949599e93c8db69d6f0a1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 21:08:04.375588 17125 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 21:08:04.375658 17105 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 21:08:04.375851 17105 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 21:08:04.377211 17105 catalog_manager.cc:1347] Generated new cluster ID: 617fb526ee284621baa3840265982d7d
I20260812 21:08:04.377255 17105 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 21:08:04.403280 17105 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 21:08:04.403482 17105 catalog_manager.cc:1504] Loading token signing keys...
I20260812 21:08:04.412433 17105 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 45ecf321ffe949599e93c8db69d6f0a1: Generated new TSK 0
I20260812 21:08:04.412688 17105 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 21:08:04.418309 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 16938
2026-08-12T21:08:04Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1480 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786568607354921-12282-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 21:08:04 dist-test-slave-vpvm krb5kdc[17133](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 21:08:04 dist-test-slave-vpvm krb5kdc[17133](info): set up 2 sockets
Aug 12 21:08:04 dist-test-slave-vpvm krb5kdc[17133](info): commencing operation
krb5kdc: starting...
W20260812 21:08:05.609236 12282 mini_kdc.cc:121] Time spent starting KDC: real 1.161s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786568607354921-12282-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786568607354921-12282-0/krb5kdc/test-user.keytab.
Aug 12 21:08:05 dist-test-slave-vpvm krb5kdc[17133](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568885, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T21:08:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T21:08:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.254.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.254.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.254.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.190.keytab.
Entry for principal kudu/127.11.254.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.190.keytab.
I20260812 21:08:05.738123 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786568607354921-12282-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786568607354921-12282-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786568607354921-12282-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.254.190:46479
--webserver_interface=127.11.254.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.190.keytab
--principal=kudu/127.11.254.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.254.148:46133
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.254.190:46479
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786568607354921-12282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786568607354921-12282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786568607354921-12282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:08:05.821812 17162 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:08:05.822062 17162 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:08:05.822085 17162 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:08:05.823288 17162 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 21:08:05.823349 17162 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:08:05.823369 17162 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:08:05.823390 17162 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 21:08:05.823413 17162 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 21:08:05.824946 17162 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.254.148:46133
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786568607354921-12282-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786568607354921-12282-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.254.190:46479
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.190.keytab
--principal=kudu/127.11.254.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.254.190:46479
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.254.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17162
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786568607354921-12282-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:08:05.825177 17162 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:08:05.825402 17162 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:08:05.830363 17177 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:08:05.830366 17173 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:08:05.830613 17162 server_base.cc:1034] running on GCE node
W20260812 21:08:05.830339 17172 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:08:05.830958 17162 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:08:05.831249 17162 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:08:05.832393 17162 hybrid_clock.cc:648] HybridClock initialized: now 1786568885832375 us; error 36 us; skew 500 ppm
Aug 12 21:08:05 dist-test-slave-vpvm krb5kdc[17133](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568885, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:08:05.835035 17162 init.cc:377] Logged in from keytab as kudu/127.11.254.190@KRBTEST.COM (short username kudu)
I20260812 21:08:05.835465 17162 webserver.cc:460] Webserver started at http://127.11.254.190:33903/ using document root <none> and password file <none>
I20260812 21:08:05.835696 17162 fs_manager.cc:359] Metadata directory not provided
I20260812 21:08:05.835748 17162 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:08:05.835876 17162 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:08:05.836961 17162 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786568607354921-12282-0/minicluster-data/master-0/data/instance:
uuid: "cf60e472e2cc4506bf6d2d18b637bb36"
format_stamp: "Formatted at 2026-08-12 21:08:05 on dist-test-slave-vpvm"
server_key: "14ba61fbe2d8278e8a2e560d1a11af73"
server_key_iv: "465ccf453984f51f0f3c03329d91afb9"
server_key_version: "encryptionkey@0"
I20260812 21:08:05.837309 17162 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786568607354921-12282-0/minicluster-data/master-0/wal/instance:
uuid: "cf60e472e2cc4506bf6d2d18b637bb36"
format_stamp: "Formatted at 2026-08-12 21:08:05 on dist-test-slave-vpvm"
server_key: "14ba61fbe2d8278e8a2e560d1a11af73"
server_key_iv: "465ccf453984f51f0f3c03329d91afb9"
server_key_version: "encryptionkey@0"
I20260812 21:08:05.838688 17162 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 21:08:05.839495 17195 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:08:05.839653 17162 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 21:08:05.839745 17162 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786568607354921-12282-0/minicluster-data/master-0/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786568607354921-12282-0/minicluster-data/master-0/wal
uuid: "cf60e472e2cc4506bf6d2d18b637bb36"
format_stamp: "Formatted at 2026-08-12 21:08:05 on dist-test-slave-vpvm"
server_key: "14ba61fbe2d8278e8a2e560d1a11af73"
server_key_iv: "465ccf453984f51f0f3c03329d91afb9"
server_key_version: "encryptionkey@0"
I20260812 21:08:05.839809 17162 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786568607354921-12282-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786568607354921-12282-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786568607354921-12282-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:08:05.861047 17162 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:08:05.863171 17162 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:08:05.863320 17162 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:08:05.867274 17162 rpc_server.cc:307] RPC server started. Bound to: 127.11.254.190:46479
I20260812 21:08:05.867312 17290 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.254.190:46479 every 8 connection(s)
I20260812 21:08:05.867656 17162 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
I20260812 21:08:05.868558 17291 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:08:05.870939 17291 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cf60e472e2cc4506bf6d2d18b637bb36: Bootstrap starting.
I20260812 21:08:05.871481 17291 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cf60e472e2cc4506bf6d2d18b637bb36: Neither blocks nor log segments found. Creating new log.
I20260812 21:08:05.871716 17291 log.cc:824] T 00000000000000000000000000000000 P cf60e472e2cc4506bf6d2d18b637bb36: Log is configured to *not* fsync() on all Append() calls
I20260812 21:08:05.872393 17291 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cf60e472e2cc4506bf6d2d18b637bb36: No bootstrap required, opened a new log
I20260812 21:08:05.874145 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 17162
I20260812 21:08:05.874065 17291 raft_consensus.cc:348] T 00000000000000000000000000000000 P cf60e472e2cc4506bf6d2d18b637bb36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf60e472e2cc4506bf6d2d18b637bb36" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 46479 } }
I20260812 21:08:05.874207 17291 raft_consensus.cc:374] T 00000000000000000000000000000000 P cf60e472e2cc4506bf6d2d18b637bb36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:08:05.874231 17291 raft_consensus.cc:729] T 00000000000000000000000000000000 P cf60e472e2cc4506bf6d2d18b637bb36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf60e472e2cc4506bf6d2d18b637bb36, State: Initialized, Role: FOLLOWER
I20260812 21:08:05.874253 12282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786568607354921-12282-0/minicluster-data/master-0/wal/instance
I20260812 21:08:05.874399 12282 external_mini_cluster.cc:1428] Setting key 3e904bd1c8f20da4a0047c27303b8559
I20260812 21:08:05.874434 17291 consensus_queue.cc:260] T 00000000000000000000000000000000 P cf60e472e2cc4506bf6d2d18b637bb36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf60e472e2cc4506bf6d2d18b637bb36" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 46479 } }
I20260812 21:08:05.874779 17294 raft_consensus.cc:482] T 00000000000000000000000000000000 P cf60e472e2cc4506bf6d2d18b637bb36 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 21:08:05.874861 17294 raft_consensus.cc:504] T 00000000000000000000000000000000 P cf60e472e2cc4506bf6d2d18b637bb36 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf60e472e2cc4506bf6d2d18b637bb36" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 46479 } }
I20260812 21:08:05.874547 17291 raft_consensus.cc:388] T 00000000000000000000000000000000 P cf60e472e2cc4506bf6d2d18b637bb36 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 21:08:05.875005 17294 leader_election.cc:302] T 00000000000000000000000000000000 P cf60e472e2cc4506bf6d2d18b637bb36 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cf60e472e2cc4506bf6d2d18b637bb36; no voters:
I20260812 21:08:05.875100 17291 raft_consensus.cc:482] T 00000000000000000000000000000000 P cf60e472e2cc4506bf6d2d18b637bb36 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 21:08:05.875144 17291 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cf60e472e2cc4506bf6d2d18b637bb36 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:08:05.875216 17294 leader_election.cc:288] T 00000000000000000000000000000000 P cf60e472e2cc4506bf6d2d18b637bb36 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers
I20260812 21:08:05.875779 17291 raft_consensus.cc:504] T 00000000000000000000000000000000 P cf60e472e2cc4506bf6d2d18b637bb36 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf60e472e2cc4506bf6d2d18b637bb36" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 46479 } }
I20260812 21:08:05.875871 17291 leader_election.cc:302] T 00000000000000000000000000000000 P cf60e472e2cc4506bf6d2d18b637bb36 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cf60e472e2cc4506bf6d2d18b637bb36; no voters:
I20260812 21:08:05.875896 17294 raft_consensus.cc:2741] T 00000000000000000000000000000000 P cf60e472e2cc4506bf6d2d18b637bb36 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260812 21:08:05.875980 17291 leader_election.cc:288] T 00000000000000000000000000000000 P cf60e472e2cc4506bf6d2d18b637bb36 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 21:08:05.876066 17294 sys_catalog.cc:455] T 00000000000000000000000000000000 P cf60e472e2cc4506bf6d2d18b637bb36 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf60e472e2cc4506bf6d2d18b637bb36" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 46479 } } }
I20260812 21:08:05.876137 17294 sys_catalog.cc:458] T 00000000000000000000000000000000 P cf60e472e2cc4506bf6d2d18b637bb36 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 21:08:05.876226 17297 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cf60e472e2cc4506bf6d2d18b637bb36 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:08:05.876257 17291 sys_catalog.cc:564] T 00000000000000000000000000000000 P cf60e472e2cc4506bf6d2d18b637bb36 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 21:08:05.876292 17297 raft_consensus.cc:686] T 00000000000000000000000000000000 P cf60e472e2cc4506bf6d2d18b637bb36 [term 1 LEADER]: Becoming Leader. State: Replica: cf60e472e2cc4506bf6d2d18b637bb36, State: Running, Role: LEADER
I20260812 21:08:05.876446 17297 consensus_queue.cc:237] T 00000000000000000000000000000000 P cf60e472e2cc4506bf6d2d18b637bb36 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf60e472e2cc4506bf6d2d18b637bb36" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 46479 } }
I20260812 21:08:05.876786 17294 sys_catalog.cc:455] T 00000000000000000000000000000000 P cf60e472e2cc4506bf6d2d18b637bb36 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cf60e472e2cc4506bf6d2d18b637bb36. Latest consensus state: current_term: 1 leader_uuid: "cf60e472e2cc4506bf6d2d18b637bb36" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf60e472e2cc4506bf6d2d18b637bb36" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 46479 } } }
I20260812 21:08:05.876847 17294 sys_catalog.cc:458] T 00000000000000000000000000000000 P cf60e472e2cc4506bf6d2d18b637bb36 [sys.catalog]: This master's current role is: LEADER
I20260812 21:08:05.877501 17312 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 21:08:05.878087 17312 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 21:08:05.879815 17312 catalog_manager.cc:1347] Generated new cluster ID: f4936a1540eb4364a0dae6aaa177fc99
I20260812 21:08:05.879866 17312 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 21:08:05 dist-test-slave-vpvm krb5kdc[17133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568885, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.254.190@KRBTEST.COM
I20260812 21:08:05.886693 17296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:05.874934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35895 (local address 127.11.254.190:46479)
0812 21:08:05.875352 (+ 418us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:05.875361 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:05.875396 (+ 35us) server_negotiation.cc:407] Connection header received
0812 21:08:05.875500 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:05.875506 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:05.875552 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:05.875664 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:05.876563 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:05.878445 (+ 1882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:05.879373 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:05.879934 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:05.880054 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:05.882827 (+ 2773us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:08:05.882831 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:08:05.882832 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:08:05.882843 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:08:05.884743 (+ 1900us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:05.885122 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:05.885125 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:05.885125 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:05.885245 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:05.885408 (+ 163us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:05.885409 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:05.885409 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:05.885708 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:08:05.885840 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:05.886322 (+ 482us) server_negotiation.cc:299] Negotiation successful
0812 21:08:05.886468 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":368,"thread_start_us":79,"threads_started":1}
I20260812 21:08:05.887668 12282 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 21:08:05.888445 17312 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 21:08:05.889084 17312 catalog_manager.cc:1504] Loading token signing keys...
I20260812 21:08:05.893275 17312 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cf60e472e2cc4506bf6d2d18b637bb36: Generated new TSK 0
I20260812 21:08:05.893451 17312 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 21:08:05.895838 17296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:05.889182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57501 (local address 127.11.254.190:46479)
0812 21:08:05.889217 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:05.889219 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:05.889330 (+ 111us) server_negotiation.cc:407] Connection header received
0812 21:08:05.889386 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:05.889387 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:05.889397 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:05.889451 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:05.890075 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:05.890809 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:05.891851 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:05.892152 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:05.892203 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:05.893595 (+ 1392us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:08:05.893597 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:08:05.893597 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:08:05.893606 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:08:05.894854 (+ 1248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:05.895225 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:05.895226 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:05.895226 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:05.895271 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:05.895440 (+ 169us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:05.895440 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:05.895441 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:05.895528 (+ 87us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:08:05.895592 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:05.895719 (+ 127us) server_negotiation.cc:299] Negotiation successful
0812 21:08:05.895779 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260812 21:08:05.897559 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 17162
2026-08-12T21:08:05Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1488 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786568607354921-12282-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 21:08:05 dist-test-slave-vpvm krb5kdc[17346](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 21:08:05 dist-test-slave-vpvm krb5kdc[17346](info): set up 2 sockets
Aug 12 21:08:05 dist-test-slave-vpvm krb5kdc[17346](info): commencing operation
krb5kdc: starting...
W20260812 21:08:07.111764 12282 mini_kdc.cc:121] Time spent starting KDC: real 1.175s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786568607354921-12282-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786568607354921-12282-0/krb5kdc/test-user.keytab.
Aug 12 21:08:07 dist-test-slave-vpvm krb5kdc[17346](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568887, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T21:08:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T21:08:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.254.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.254.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.254.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.190.keytab.
Entry for principal kudu/127.11.254.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.190.keytab.
I20260812 21:08:07.241096 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786568607354921-12282-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786568607354921-12282-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786568607354921-12282-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.254.190:41445
--webserver_interface=127.11.254.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.190.keytab
--principal=kudu/127.11.254.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.254.148:42759
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.254.190:41445
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786568607354921-12282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786568607354921-12282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786568607354921-12282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:08:07.324968 17368 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:08:07.325197 17368 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:08:07.325229 17368 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:08:07.326435 17368 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 21:08:07.326480 17368 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:08:07.326501 17368 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:08:07.326516 17368 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 21:08:07.326534 17368 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 21:08:07.328095 17368 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.254.148:42759
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786568607354921-12282-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786568607354921-12282-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.254.190:41445
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.190.keytab
--principal=kudu/127.11.254.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.254.190:41445
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.254.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17368
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786568607354921-12282-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:08:07.328344 17368 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:08:07.328606 17368 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:08:07.333668 17379 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:08:07.333676 17383 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:08:07.333858 17368 server_base.cc:1034] running on GCE node
W20260812 21:08:07.333719 17378 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:08:07.334168 17368 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:08:07.334506 17368 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:08:07.335660 17368 hybrid_clock.cc:648] HybridClock initialized: now 1786568887335645 us; error 30 us; skew 500 ppm
Aug 12 21:08:07 dist-test-slave-vpvm krb5kdc[17346](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568887, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:08:07.342099 17368 init.cc:377] Logged in from keytab as kudu/127.11.254.190@KRBTEST.COM (short username kudu)
I20260812 21:08:07.342594 17368 webserver.cc:460] Webserver started at http://127.11.254.190:33995/ using document root <none> and password file <none>
I20260812 21:08:07.342832 17368 fs_manager.cc:359] Metadata directory not provided
I20260812 21:08:07.342880 17368 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:08:07.342999 17368 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:08:07.344136 17368 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786568607354921-12282-0/minicluster-data/master-0/data/instance:
uuid: "eaeeafb2fa7848dea80e98e5217caf9c"
format_stamp: "Formatted at 2026-08-12 21:08:07 on dist-test-slave-vpvm"
server_key: "eb2839ba4c5489a09493c179548578ef"
server_key_iv: "c8a2b54bd71282bb91b06295114c573f"
server_key_version: "encryptionkey@0"
I20260812 21:08:07.344481 17368 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786568607354921-12282-0/minicluster-data/master-0/wal/instance:
uuid: "eaeeafb2fa7848dea80e98e5217caf9c"
format_stamp: "Formatted at 2026-08-12 21:08:07 on dist-test-slave-vpvm"
server_key: "eb2839ba4c5489a09493c179548578ef"
server_key_iv: "c8a2b54bd71282bb91b06295114c573f"
server_key_version: "encryptionkey@0"
I20260812 21:08:07.345826 17368 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 21:08:07.346632 17396 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:08:07.346794 17368 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 21:08:07.346886 17368 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786568607354921-12282-0/minicluster-data/master-0/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786568607354921-12282-0/minicluster-data/master-0/wal
uuid: "eaeeafb2fa7848dea80e98e5217caf9c"
format_stamp: "Formatted at 2026-08-12 21:08:07 on dist-test-slave-vpvm"
server_key: "eb2839ba4c5489a09493c179548578ef"
server_key_iv: "c8a2b54bd71282bb91b06295114c573f"
server_key_version: "encryptionkey@0"
I20260812 21:08:07.346972 17368 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786568607354921-12282-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786568607354921-12282-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786568607354921-12282-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:08:07.376014 17368 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:08:07.378115 17368 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:08:07.378259 17368 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:08:07.382166 17368 rpc_server.cc:307] RPC server started. Bound to: 127.11.254.190:41445
I20260812 21:08:07.382221 17484 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.254.190:41445 every 8 connection(s)
I20260812 21:08:07.382550 17368 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
I20260812 21:08:07.383425 17487 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:08:07.385805 17487 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eaeeafb2fa7848dea80e98e5217caf9c: Bootstrap starting.
I20260812 21:08:07.386445 17487 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eaeeafb2fa7848dea80e98e5217caf9c: Neither blocks nor log segments found. Creating new log.
I20260812 21:08:07.386745 17487 log.cc:824] T 00000000000000000000000000000000 P eaeeafb2fa7848dea80e98e5217caf9c: Log is configured to *not* fsync() on all Append() calls
I20260812 21:08:07.387257 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 17368
I20260812 21:08:07.387373 12282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786568607354921-12282-0/minicluster-data/master-0/wal/instance
I20260812 21:08:07.387534 17487 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eaeeafb2fa7848dea80e98e5217caf9c: No bootstrap required, opened a new log
I20260812 21:08:07.387538 12282 external_mini_cluster.cc:1428] Setting key c1021390667ea38abeb9eb537eaf52c5
I20260812 21:08:07.389526 17487 raft_consensus.cc:348] T 00000000000000000000000000000000 P eaeeafb2fa7848dea80e98e5217caf9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eaeeafb2fa7848dea80e98e5217caf9c" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 41445 } }
I20260812 21:08:07.389684 17487 raft_consensus.cc:374] T 00000000000000000000000000000000 P eaeeafb2fa7848dea80e98e5217caf9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:08:07.389716 17487 raft_consensus.cc:729] T 00000000000000000000000000000000 P eaeeafb2fa7848dea80e98e5217caf9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eaeeafb2fa7848dea80e98e5217caf9c, State: Initialized, Role: FOLLOWER
I20260812 21:08:07.389853 17487 consensus_queue.cc:260] T 00000000000000000000000000000000 P eaeeafb2fa7848dea80e98e5217caf9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eaeeafb2fa7848dea80e98e5217caf9c" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 41445 } }
I20260812 21:08:07.389930 17487 raft_consensus.cc:388] T 00000000000000000000000000000000 P eaeeafb2fa7848dea80e98e5217caf9c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 21:08:07.389987 17487 raft_consensus.cc:482] T 00000000000000000000000000000000 P eaeeafb2fa7848dea80e98e5217caf9c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 21:08:07.390038 17487 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eaeeafb2fa7848dea80e98e5217caf9c [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:08:07.390661 17487 raft_consensus.cc:504] T 00000000000000000000000000000000 P eaeeafb2fa7848dea80e98e5217caf9c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eaeeafb2fa7848dea80e98e5217caf9c" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 41445 } }
I20260812 21:08:07.390769 17487 leader_election.cc:302] T 00000000000000000000000000000000 P eaeeafb2fa7848dea80e98e5217caf9c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: eaeeafb2fa7848dea80e98e5217caf9c; no voters:
I20260812 21:08:07.390931 17487 leader_election.cc:288] T 00000000000000000000000000000000 P eaeeafb2fa7848dea80e98e5217caf9c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 21:08:07.390972 17494 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eaeeafb2fa7848dea80e98e5217caf9c [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:08:07.391127 17494 raft_consensus.cc:686] T 00000000000000000000000000000000 P eaeeafb2fa7848dea80e98e5217caf9c [term 1 LEADER]: Becoming Leader. State: Replica: eaeeafb2fa7848dea80e98e5217caf9c, State: Running, Role: LEADER
I20260812 21:08:07.391223 17487 sys_catalog.cc:564] T 00000000000000000000000000000000 P eaeeafb2fa7848dea80e98e5217caf9c [sys.catalog]: configured and running, proceeding with master startup.
I20260812 21:08:07.391268 17494 consensus_queue.cc:237] T 00000000000000000000000000000000 P eaeeafb2fa7848dea80e98e5217caf9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eaeeafb2fa7848dea80e98e5217caf9c" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 41445 } }
I20260812 21:08:07.391686 17494 sys_catalog.cc:455] T 00000000000000000000000000000000 P eaeeafb2fa7848dea80e98e5217caf9c [sys.catalog]: SysCatalogTable state changed. Reason: New leader eaeeafb2fa7848dea80e98e5217caf9c. Latest consensus state: current_term: 1 leader_uuid: "eaeeafb2fa7848dea80e98e5217caf9c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eaeeafb2fa7848dea80e98e5217caf9c" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 41445 } } }
I20260812 21:08:07.391916 17494 sys_catalog.cc:458] T 00000000000000000000000000000000 P eaeeafb2fa7848dea80e98e5217caf9c [sys.catalog]: This master's current role is: LEADER
I20260812 21:08:07.392696 17498 sys_catalog.cc:455] T 00000000000000000000000000000000 P eaeeafb2fa7848dea80e98e5217caf9c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "eaeeafb2fa7848dea80e98e5217caf9c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eaeeafb2fa7848dea80e98e5217caf9c" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 41445 } } }
I20260812 21:08:07.392776 17498 sys_catalog.cc:458] T 00000000000000000000000000000000 P eaeeafb2fa7848dea80e98e5217caf9c [sys.catalog]: This master's current role is: LEADER
W20260812 21:08:07.393002 17514 catalog_manager.cc:1558] T 00000000000000000000000000000000 P eaeeafb2fa7848dea80e98e5217caf9c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 21:08:07.393070 17514 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 21:08:07.393150 17505 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 21:08:07.393440 17505 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 21:08:07.393774 17493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:07.388045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33291 (local address 127.11.254.190:41445)
0812 21:08:07.388260 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:07.388268 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:07.388298 (+ 30us) server_negotiation.cc:407] Connection header received
0812 21:08:07.388828 (+ 530us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:07.388835 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:07.388883 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:07.389008 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:07.389229 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:07.390563 (+ 1334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:07.391607 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:07.392180 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:07.392291 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:07.393002 (+ 711us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:08:07.393006 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:08:07.393006 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 21:08:07.393221 (+ 215us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:08:07.393262 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 21:08:07.393279 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:08:07.393346 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:07.393472 (+ 126us) server_negotiation.cc:299] Negotiation successful
0812 21:08:07.393550 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":98,"threads_started":1}
I20260812 21:08:07.394798 12282 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 21:08:07.395121 17505 catalog_manager.cc:1347] Generated new cluster ID: f8a81cc6ca5f4c3d9dd97e9fec69eb26
I20260812 21:08:07.395205 17505 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 21:08:07.399660 17493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:07.396295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40649 (local address 127.11.254.190:41445)
0812 21:08:07.396331 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:07.396333 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:07.396507 (+ 174us) server_negotiation.cc:407] Connection header received
0812 21:08:07.396535 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:07.396536 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:07.396548 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:07.396615 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:07.396716 (+ 101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:07.397447 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:07.398407 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:07.398704 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:07.398779 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:07.399430 (+ 651us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:08:07.399431 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:08:07.399432 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 21:08:07.399437 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:08:07.399455 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 21:08:07.399459 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:08:07.399516 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:07.399560 (+ 44us) server_negotiation.cc:299] Negotiation successful
0812 21:08:07.399583 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260812 21:08:07.412536 17505 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 21:08:07.413301 17505 catalog_manager.cc:1504] Loading token signing keys...
I20260812 21:08:07.423329 17505 catalog_manager.cc:5868] T 00000000000000000000000000000000 P eaeeafb2fa7848dea80e98e5217caf9c: Generated new TSK 0
I20260812 21:08:07.423537 17505 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 21:08:07.431474 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 17368
2026-08-12T21:08:07Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1527 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786568607354921-12282-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 21:08:07 dist-test-slave-vpvm krb5kdc[17546](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 21:08:07 dist-test-slave-vpvm krb5kdc[17546](info): set up 2 sockets
Aug 12 21:08:07 dist-test-slave-vpvm krb5kdc[17546](info): commencing operation
krb5kdc: starting...
W20260812 21:08:08.622987 12282 mini_kdc.cc:121] Time spent starting KDC: real 1.159s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786568607354921-12282-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786568607354921-12282-0/krb5kdc/test-user.keytab.
Aug 12 21:08:08 dist-test-slave-vpvm krb5kdc[17546](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568888, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T21:08:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T21:08:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.254.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.254.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.254.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.190.keytab.
Entry for principal kudu/127.11.254.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.190.keytab.
I20260812 21:08:08.749729 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786568607354921-12282-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786568607354921-12282-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786568607354921-12282-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.254.190:38353
--webserver_interface=127.11.254.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.190.keytab
--principal=kudu/127.11.254.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.254.148:33881
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.254.190:38353
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786568607354921-12282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786568607354921-12282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786568607354921-12282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:08:08.833057 17565 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:08:08.833272 17565 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:08:08.833294 17565 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:08:08.834412 17565 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 21:08:08.834451 17565 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:08:08.834463 17565 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:08:08.834477 17565 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 21:08:08.834488 17565 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 21:08:08.835990 17565 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.254.148:33881
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786568607354921-12282-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786568607354921-12282-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.254.190:38353
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.190.keytab
--principal=kudu/127.11.254.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.254.190:38353
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.254.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17565
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786568607354921-12282-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:08:08.836193 17565 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:08:08.836380 17565 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:08:08.841375 17577 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:08:08.841452 17575 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:08:08.841593 17565 server_base.cc:1034] running on GCE node
W20260812 21:08:08.841382 17580 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:08:08.841904 17565 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:08:08.842219 17565 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:08:08.843361 17565 hybrid_clock.cc:648] HybridClock initialized: now 1786568888843334 us; error 45 us; skew 500 ppm
Aug 12 21:08:08 dist-test-slave-vpvm krb5kdc[17546](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568888, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:08:08.845871 17565 init.cc:377] Logged in from keytab as kudu/127.11.254.190@KRBTEST.COM (short username kudu)
I20260812 21:08:08.846328 17565 webserver.cc:460] Webserver started at http://127.11.254.190:33587/ using document root <none> and password file <none>
I20260812 21:08:08.846539 17565 fs_manager.cc:359] Metadata directory not provided
I20260812 21:08:08.846589 17565 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:08:08.846716 17565 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:08:08.847785 17565 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786568607354921-12282-0/minicluster-data/master-0/data/instance:
uuid: "8839f01372094215a7748269ad4f1178"
format_stamp: "Formatted at 2026-08-12 21:08:08 on dist-test-slave-vpvm"
server_key: "1635735422b539dddadff115c56a1b0f"
server_key_iv: "d34e693c57c7372772b96d779345096e"
server_key_version: "encryptionkey@0"
I20260812 21:08:08.848142 17565 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786568607354921-12282-0/minicluster-data/master-0/wal/instance:
uuid: "8839f01372094215a7748269ad4f1178"
format_stamp: "Formatted at 2026-08-12 21:08:08 on dist-test-slave-vpvm"
server_key: "1635735422b539dddadff115c56a1b0f"
server_key_iv: "d34e693c57c7372772b96d779345096e"
server_key_version: "encryptionkey@0"
I20260812 21:08:08.849547 17565 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 21:08:08.850330 17592 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:08:08.850509 17565 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 21:08:08.850602 17565 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786568607354921-12282-0/minicluster-data/master-0/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786568607354921-12282-0/minicluster-data/master-0/wal
uuid: "8839f01372094215a7748269ad4f1178"
format_stamp: "Formatted at 2026-08-12 21:08:08 on dist-test-slave-vpvm"
server_key: "1635735422b539dddadff115c56a1b0f"
server_key_iv: "d34e693c57c7372772b96d779345096e"
server_key_version: "encryptionkey@0"
I20260812 21:08:08.850669 17565 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786568607354921-12282-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786568607354921-12282-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786568607354921-12282-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:08:08.854063 17565 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:08:08.856040 17565 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:08:08.856192 17565 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:08:08.860086 17565 rpc_server.cc:307] RPC server started. Bound to: 127.11.254.190:38353
I20260812 21:08:08.860117 17680 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.254.190:38353 every 8 connection(s)
I20260812 21:08:08.860435 17565 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
I20260812 21:08:08.861407 17681 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:08:08.863767 17681 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8839f01372094215a7748269ad4f1178: Bootstrap starting.
I20260812 21:08:08.864308 17681 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8839f01372094215a7748269ad4f1178: Neither blocks nor log segments found. Creating new log.
I20260812 21:08:08.864542 17681 log.cc:824] T 00000000000000000000000000000000 P 8839f01372094215a7748269ad4f1178: Log is configured to *not* fsync() on all Append() calls
I20260812 21:08:08.865295 17681 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8839f01372094215a7748269ad4f1178: No bootstrap required, opened a new log
I20260812 21:08:08.865300 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 17565
I20260812 21:08:08.865391 12282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786568607354921-12282-0/minicluster-data/master-0/wal/instance
I20260812 21:08:08.865592 12282 external_mini_cluster.cc:1428] Setting key 3c1f597e089f13f7f0f5db3fef403125
I20260812 21:08:08.867419 17681 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8839f01372094215a7748269ad4f1178 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8839f01372094215a7748269ad4f1178" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 38353 } }
I20260812 21:08:08.867610 17681 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8839f01372094215a7748269ad4f1178 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 21:08:08.867650 17681 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8839f01372094215a7748269ad4f1178 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8839f01372094215a7748269ad4f1178, State: Initialized, Role: FOLLOWER
I20260812 21:08:08.867817 17681 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8839f01372094215a7748269ad4f1178 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8839f01372094215a7748269ad4f1178" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 38353 } }
I20260812 21:08:08.867911 17681 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8839f01372094215a7748269ad4f1178 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 21:08:08.867940 17681 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8839f01372094215a7748269ad4f1178 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 21:08:08.867985 17681 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8839f01372094215a7748269ad4f1178 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:08:08.868850 17692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:08.866151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50487 (local address 127.11.254.190:38353)
0812 21:08:08.866352 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:08.866362 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:08.866398 (+ 36us) server_negotiation.cc:407] Connection header received
0812 21:08:08.867378 (+ 980us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:08.867387 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:08.867426 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:08.867594 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:08.868285 (+ 691us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:08:08.868288 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:08:08.868288 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 21:08:08.868293 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:08:08.868347 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 21:08:08.868361 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:08:08.868434 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:08.868579 (+ 145us) server_negotiation.cc:299] Negotiation successful
0812 21:08:08.868625 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":99,"threads_started":1}
I20260812 21:08:08.869107 17681 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8839f01372094215a7748269ad4f1178 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8839f01372094215a7748269ad4f1178" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 38353 } }
I20260812 21:08:08.869212 17681 leader_election.cc:302] T 00000000000000000000000000000000 P 8839f01372094215a7748269ad4f1178 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8839f01372094215a7748269ad4f1178; no voters:
I20260812 21:08:08.869377 17681 leader_election.cc:288] T 00000000000000000000000000000000 P 8839f01372094215a7748269ad4f1178 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 21:08:08.869403 17694 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8839f01372094215a7748269ad4f1178 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:08:08.869686 17681 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8839f01372094215a7748269ad4f1178 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 21:08:08.869733 17694 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8839f01372094215a7748269ad4f1178 [term 1 LEADER]: Becoming Leader. State: Replica: 8839f01372094215a7748269ad4f1178, State: Running, Role: LEADER
I20260812 21:08:08.869886 12282 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 21:08:08.869871 17694 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8839f01372094215a7748269ad4f1178 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8839f01372094215a7748269ad4f1178" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 38353 } }
I20260812 21:08:08.870244 17699 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8839f01372094215a7748269ad4f1178 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8839f01372094215a7748269ad4f1178. Latest consensus state: current_term: 1 leader_uuid: "8839f01372094215a7748269ad4f1178" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8839f01372094215a7748269ad4f1178" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 38353 } } }
I20260812 21:08:08.870321 17699 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8839f01372094215a7748269ad4f1178 [sys.catalog]: This master's current role is: LEADER
I20260812 21:08:08.870517 17695 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8839f01372094215a7748269ad4f1178 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8839f01372094215a7748269ad4f1178" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8839f01372094215a7748269ad4f1178" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 38353 } } }
I20260812 21:08:08.870623 17695 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8839f01372094215a7748269ad4f1178 [sys.catalog]: This master's current role is: LEADER
I20260812 21:08:08.871037 17704 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 21:08:08.871636 17704 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 21:08:08.872630 17692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:08.871454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50079 (local address 127.11.254.190:38353)
0812 21:08:08.871487 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:08.871489 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:08.871618 (+ 129us) server_negotiation.cc:407] Connection header received
0812 21:08:08.871682 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:08.871683 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:08.871689 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:08.871744 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:08.872407 (+ 663us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:08:08.872409 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:08:08.872409 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 21:08:08.872413 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:08:08.872431 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 21:08:08.872435 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:08:08.872464 (+ 29us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:08.872556 (+ 92us) server_negotiation.cc:299] Negotiation successful
0812 21:08:08.872568 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260812 21:08:08.873267 17704 catalog_manager.cc:1347] Generated new cluster ID: b743f90db3d94bcf9d5f796e894922ec
I20260812 21:08:08.873317 17704 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 21:08:08.880385 17704 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 21:08:08.880568 17704 catalog_manager.cc:1504] Loading token signing keys...
I20260812 21:08:08.882411 17704 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8839f01372094215a7748269ad4f1178: Generated new TSK 0
I20260812 21:08:08.882580 17704 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 21:08:08.888546 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 17565
2026-08-12T21:08:08Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1472 ms)
[----------] 18 tests from AuthTokenIssuingTest (26715 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786568607354921-12282-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 21:08:08 dist-test-slave-vpvm krb5kdc[17736](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 21:08:08 dist-test-slave-vpvm krb5kdc[17736](info): set up 2 sockets
Aug 12 21:08:08 dist-test-slave-vpvm krb5kdc[17736](info): commencing operation
krb5kdc: starting...
W20260812 21:08:10.092063 12282 mini_kdc.cc:121] Time spent starting KDC: real 1.156s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786568607354921-12282-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786568607354921-12282-0/krb5kdc/test-user.keytab.
Aug 12 21:08:10 dist-test-slave-vpvm krb5kdc[17736](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568890, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T21:08:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T21:08:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.254.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.254.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.254.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.190.keytab.
Entry for principal kudu/127.11.254.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.190.keytab.
I20260812 21:08:10.220995 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786568607354921-12282-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786568607354921-12282-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786568607354921-12282-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.254.190:42299
--webserver_interface=127.11.254.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.190.keytab
--principal=kudu/127.11.254.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.254.148:41483
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.254.190:42299,127.11.254.189:45185,127.11.254.188:33803
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786568607354921-12282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786568607354921-12282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786568607354921-12282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:08:10.308427 17761 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 21:08:10.308674 17761 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:08:10.308701 17761 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:08:10.308728 17761 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:08:10.310148 17761 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 21:08:10.310210 17761 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:08:10.310232 17761 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:08:10.310251 17761 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 21:08:10.310266 17761 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 21:08:10.312022 17761 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.254.148:41483
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786568607354921-12282-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786568607354921-12282-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.254.190:42299,127.11.254.189:45185,127.11.254.188:33803
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.190.keytab
--principal=kudu/127.11.254.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.254.190:42299
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.254.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17761
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786568607354921-12282-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:08:10.312310 17761 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:08:10.312551 17761 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:08:10.317646 17774 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:08:10.317672 17768 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:08:10.317862 17761 server_base.cc:1034] running on GCE node
W20260812 21:08:10.317729 17769 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:08:10.318223 17761 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:08:10.318552 17761 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:08:10.319732 17761 hybrid_clock.cc:648] HybridClock initialized: now 1786568890319709 us; error 38 us; skew 500 ppm
Aug 12 21:08:10 dist-test-slave-vpvm krb5kdc[17736](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568890, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:08:10.322633 17761 init.cc:377] Logged in from keytab as kudu/127.11.254.190@KRBTEST.COM (short username kudu)
I20260812 21:08:10.323182 17761 webserver.cc:460] Webserver started at http://127.11.254.190:35697/ using document root <none> and password file <none>
I20260812 21:08:10.323463 17761 fs_manager.cc:359] Metadata directory not provided
I20260812 21:08:10.323521 17761 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:08:10.323678 17761 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:08:10.324856 17761 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786568607354921-12282-0/minicluster-data/master-0/data/instance:
uuid: "2d02fea4ccda4d95a55e980128dc75ea"
format_stamp: "Formatted at 2026-08-12 21:08:10 on dist-test-slave-vpvm"
server_key: "f10ee733880154dc76bfcb58c23ead02"
server_key_iv: "84a28e9fe6fb8088754dc96bcc5f3a55"
server_key_version: "encryptionkey@0"
I20260812 21:08:10.325224 17761 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786568607354921-12282-0/minicluster-data/master-0/wal/instance:
uuid: "2d02fea4ccda4d95a55e980128dc75ea"
format_stamp: "Formatted at 2026-08-12 21:08:10 on dist-test-slave-vpvm"
server_key: "f10ee733880154dc76bfcb58c23ead02"
server_key_iv: "84a28e9fe6fb8088754dc96bcc5f3a55"
server_key_version: "encryptionkey@0"
I20260812 21:08:10.326663 17761 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 21:08:10.327606 17781 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:08:10.327827 17761 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 21:08:10.327946 17761 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786568607354921-12282-0/minicluster-data/master-0/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786568607354921-12282-0/minicluster-data/master-0/wal
uuid: "2d02fea4ccda4d95a55e980128dc75ea"
format_stamp: "Formatted at 2026-08-12 21:08:10 on dist-test-slave-vpvm"
server_key: "f10ee733880154dc76bfcb58c23ead02"
server_key_iv: "84a28e9fe6fb8088754dc96bcc5f3a55"
server_key_version: "encryptionkey@0"
I20260812 21:08:10.328017 17761 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786568607354921-12282-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786568607354921-12282-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786568607354921-12282-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:08:10.338624 17761 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:08:10.340953 17761 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:08:10.341102 17761 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:08:10.345278 17761 rpc_server.cc:307] RPC server started. Bound to: 127.11.254.190:42299
I20260812 21:08:10.345311 17873 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.254.190:42299 every 8 connection(s)
I20260812 21:08:10.345674 17761 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
I20260812 21:08:10.346603 17874 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:08:10.346835 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 17761
I20260812 21:08:10.346946 12282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786568607354921-12282-0/minicluster-data/master-0/wal/instance
I20260812 21:08:10.347167 12282 external_mini_cluster.cc:1428] Setting key db24cd19a22b7ef65c95e172e8148728
I20260812 21:08:10.348762 17874 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.254.190" port: 42299 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 21:08:10 dist-test-slave-vpvm krb5kdc[17736](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568890, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.190@KRBTEST.COM for kudu/127.11.254.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.11.254.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.254.189@KRBTEST.COM" created.
I20260812 21:08:10.359972 17880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:10.349557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:42299 (local address 127.0.0.1:58512)
0812 21:08:10.349732 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:10.349778 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 21:08:10.350326 (+ 548us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:08:10.350667 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:08:10.350669 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:08:10.350940 (+ 271us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:08:10.351131 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:10.351135 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:10.352682 (+ 1547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:10.352683 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:10.353375 (+ 692us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:10.353377 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:10.354074 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:10.354075 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:10.354155 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:10.354707 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:08:10.354717 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:08:10.356166 (+ 1449us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:08:10.358221 (+ 2055us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:08:10.358229 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:08:10.358231 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:08:10.358433 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:08:10.358799 (+ 366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:08:10.358799 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:08:10.358800 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:08:10.358912 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:08:10.359385 (+ 473us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:08:10.359392 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:08:10.359461 (+ 69us) client_negotiation.cc:769] Sending connection context
0812 21:08:10.359558 (+ 97us) client_negotiation.cc:240] Negotiation successful
0812 21:08:10.359717 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":69,"threads_started":1}
I20260812 21:08:10.359980 17882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:10.349744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58512 (local address 127.11.254.190:42299)
0812 21:08:10.349892 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:10.349897 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:10.349966 (+ 69us) server_negotiation.cc:407] Connection header received
0812 21:08:10.350527 (+ 561us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:10.350529 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:10.350553 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:10.350658 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:10.351236 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:10.352578 (+ 1342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:10.353467 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:10.353983 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:10.354095 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:10.356319 (+ 2224us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:08:10.356327 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:08:10.356327 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:08:10.356334 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:08:10.358143 (+ 1809us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:10.358541 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:10.358544 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:10.358545 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:10.358720 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:10.359005 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:10.359006 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:10.359007 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:10.359281 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:08:10.359381 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:10.359614 (+ 233us) server_negotiation.cc:299] Negotiation successful
0812 21:08:10.359791 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":67,"server-negotiator.queue_time_us":112,"thread_start_us":69,"threads_started":1}
I20260812 21:08:10.360802 17874 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.254.189" port: 45185 } has no permanent_uuid. Determining permanent_uuid...
I20260812 21:08:10.361198 17880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:10.361074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.189:45185 (local address 127.0.0.1:60294)
0812 21:08:10.361127 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:10.361179 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.189:45185: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260812 21:08:10.361343 17782 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.254.189:45185: connect: Connection refused (error 111)
I20260812 21:08:10.361809 17880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:10.361717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.189:45185 (local address 127.0.0.1:60306)
0812 21:08:10.361760 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:10.361786 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.189:45185: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260812 21:08:10.361907 17874 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.254.189:45185: Network error: Client connection negotiation failed: client connection to 127.11.254.189:45185: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.254.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.189.keytab.
Entry for principal kudu/127.11.254.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.189.keytab.
I20260812 21:08:10.377637 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786568607354921-12282-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786568607354921-12282-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786568607354921-12282-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786568607354921-12282-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.254.189:45185
--webserver_interface=127.11.254.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.189.keytab
--principal=kudu/127.11.254.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.254.148:41483
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.254.190:42299,127.11.254.189:45185,127.11.254.188:33803
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786568607354921-12282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786568607354921-12282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786568607354921-12282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 21:08:10.411193 17874 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.254.189" port: 45185 } attempt: 1
I20260812 21:08:10.411790 17880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:10.411601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.189:45185 (local address 127.0.0.1:60312)
0812 21:08:10.411683 (+ 82us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:10.411745 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.189:45185: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260812 21:08:10.412177 17880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:10.412076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.189:45185 (local address 127.0.0.1:60320)
0812 21:08:10.412119 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:10.412152 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.189:45185: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260812 21:08:10.412325 17874 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.254.189:45185: Network error: Client connection negotiation failed: client connection to 127.11.254.189:45185: connect: Connection refused (error 111)
W20260812 21:08:10.465127 17887 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 21:08:10.465355 17887 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:08:10.465371 17887 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:08:10.465391 17887 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:08:10.466598 17887 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 21:08:10.466642 17887 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:08:10.466655 17887 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:08:10.466666 17887 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 21:08:10.466686 17887 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 21:08:10.468189 17887 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.254.148:41483
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786568607354921-12282-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786568607354921-12282-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.11.254.190:42299,127.11.254.189:45185,127.11.254.188:33803
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.189.keytab
--principal=kudu/127.11.254.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.254.189:45185
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786568607354921-12282-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.11.254.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17887
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786568607354921-12282-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:08:10.468436 17887 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:08:10.468678 17887 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:08:10.473785 17904 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:08:10.473778 17906 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:08:10.473778 17903 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:08:10.474095 17887 server_base.cc:1034] running on GCE node
I20260812 21:08:10.474357 17887 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:08:10.474655 17887 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:08:10.475813 17887 hybrid_clock.cc:648] HybridClock initialized: now 1786568890475780 us; error 48 us; skew 500 ppm
Aug 12 21:08:10 dist-test-slave-vpvm krb5kdc[17736](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568890, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:08:10.478377 17887 init.cc:377] Logged in from keytab as kudu/127.11.254.189@KRBTEST.COM (short username kudu)
I20260812 21:08:10.478878 17887 webserver.cc:460] Webserver started at http://127.11.254.189:41999/ using document root <none> and password file <none>
I20260812 21:08:10.479092 17887 fs_manager.cc:359] Metadata directory not provided
I20260812 21:08:10.479144 17887 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:08:10.479275 17887 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:08:10.480505 17874 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.254.189" port: 45185 } attempt: 2
I20260812 21:08:10.480387 17887 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786568607354921-12282-0/minicluster-data/master-1/data/instance:
uuid: "16f134d6ec644e2b90b113224a2816e2"
format_stamp: "Formatted at 2026-08-12 21:08:10 on dist-test-slave-vpvm"
server_key: "89bcab1cd23820b121ff12deecf1833f"
server_key_iv: "a7f01a11eb38a54c8bddf2c9006412b7"
server_key_version: "encryptionkey@0"
I20260812 21:08:10.480841 17887 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786568607354921-12282-0/minicluster-data/master-1/wal/instance:
uuid: "16f134d6ec644e2b90b113224a2816e2"
format_stamp: "Formatted at 2026-08-12 21:08:10 on dist-test-slave-vpvm"
server_key: "89bcab1cd23820b121ff12deecf1833f"
server_key_iv: "a7f01a11eb38a54c8bddf2c9006412b7"
server_key_version: "encryptionkey@0"
I20260812 21:08:10.480921 17880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:10.480780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.189:45185 (local address 127.0.0.1:60324)
0812 21:08:10.480842 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:10.480881 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.189:45185: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260812 21:08:10.481284 17880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:10.481178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.189:45185 (local address 127.0.0.1:60340)
0812 21:08:10.481222 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:10.481255 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.189:45185: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260812 21:08:10.481374 17874 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.254.189:45185: Network error: Client connection negotiation failed: client connection to 127.11.254.189:45185: connect: Connection refused (error 111)
I20260812 21:08:10.482355 17887 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 21:08:10.483111 17923 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:08:10.483271 17887 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 21:08:10.483356 17887 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786568607354921-12282-0/minicluster-data/master-1/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786568607354921-12282-0/minicluster-data/master-1/wal
uuid: "16f134d6ec644e2b90b113224a2816e2"
format_stamp: "Formatted at 2026-08-12 21:08:10 on dist-test-slave-vpvm"
server_key: "89bcab1cd23820b121ff12deecf1833f"
server_key_iv: "a7f01a11eb38a54c8bddf2c9006412b7"
server_key_version: "encryptionkey@0"
I20260812 21:08:10.483435 17887 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786568607354921-12282-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786568607354921-12282-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786568607354921-12282-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:08:10.496464 17887 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:08:10.498492 17887 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:08:10.498615 17887 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:08:10.502519 17887 rpc_server.cc:307] RPC server started. Bound to: 127.11.254.189:45185
I20260812 21:08:10.502578 18007 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.254.189:45185 every 8 connection(s)
I20260812 21:08:10.502930 17887 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786568607354921-12282-0/minicluster-data/master-1/data/info.pb
I20260812 21:08:10.503831 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 17887
I20260812 21:08:10.503925 12282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786568607354921-12282-0/minicluster-data/master-1/wal/instance
I20260812 21:08:10.503934 18009 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:08:10.504105 12282 external_mini_cluster.cc:1428] Setting key a3968136f8120a9b0bd538f4c6dba915
I20260812 21:08:10.505592 18009 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.254.190" port: 42299 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 21:08:10 dist-test-slave-vpvm krb5kdc[17736](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568890, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.189@KRBTEST.COM for kudu/127.11.254.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.11.254.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.254.188@KRBTEST.COM" created.
I20260812 21:08:10.515198 17882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:10.506459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58526 (local address 127.11.254.190:42299)
0812 21:08:10.506526 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:10.506529 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:10.506728 (+ 199us) server_negotiation.cc:407] Connection header received
0812 21:08:10.507365 (+ 637us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:10.507367 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:10.507383 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:10.507448 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:10.508091 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:10.508843 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:10.509924 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:10.510258 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:10.510328 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:10.512455 (+ 2127us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:08:10.512456 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:08:10.512457 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:08:10.512465 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:08:10.513673 (+ 1208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:10.514118 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:10.514119 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:10.514120 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:10.514175 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:10.514474 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:10.514474 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:10.514475 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:10.514644 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:08:10.514700 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:10.515014 (+ 314us) server_negotiation.cc:299] Negotiation successful
0812 21:08:10.515103 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260812 21:08:10.515455 18018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:10.506420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:42299 (local address 127.0.0.1:58526)
0812 21:08:10.506586 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:10.506637 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0812 21:08:10.507258 (+ 621us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:08:10.507510 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:08:10.507513 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:08:10.507766 (+ 253us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:08:10.507992 (+ 226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:10.507996 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:10.508970 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:10.508972 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:10.509832 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:10.509834 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:10.510348 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:10.510349 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:10.510476 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:10.510992 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:08:10.511001 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:08:10.512363 (+ 1362us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:08:10.513764 (+ 1401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:08:10.513771 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:08:10.513773 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:08:10.514048 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:08:10.514259 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:08:10.514260 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:08:10.514261 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:08:10.514399 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:08:10.514727 (+ 328us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:08:10.514734 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:08:10.514870 (+ 136us) client_negotiation.cc:769] Sending connection context
0812 21:08:10.514988 (+ 118us) client_negotiation.cc:240] Negotiation successful
0812 21:08:10.515148 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":79,"threads_started":1}
I20260812 21:08:10.516031 18009 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.254.189" port: 45185 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 21:08:10 dist-test-slave-vpvm krb5kdc[17736](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568890, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.189@KRBTEST.COM for kudu/127.11.254.189@KRBTEST.COM
I20260812 21:08:10.523592 18018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:10.516291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.189:45185 (local address 127.0.0.1:60348)
0812 21:08:10.516335 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:10.516347 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 21:08:10.516390 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:08:10.516643 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:08:10.516644 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:08:10.516789 (+ 145us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:08:10.516852 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:10.516853 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:10.517753 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:10.517754 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:10.518467 (+ 713us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:10.518470 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:10.518960 (+ 490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:10.518962 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:10.519036 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:10.519440 (+ 404us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:08:10.519445 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:08:10.520774 (+ 1329us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:08:10.522613 (+ 1839us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:08:10.522614 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:08:10.522614 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:08:10.522768 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:08:10.522945 (+ 177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:08:10.522946 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:08:10.522946 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:08:10.522998 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:08:10.523263 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:08:10.523264 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:08:10.523303 (+ 39us) client_negotiation.cc:769] Sending connection context
0812 21:08:10.523365 (+ 62us) client_negotiation.cc:240] Negotiation successful
0812 21:08:10.523475 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25}
I20260812 21:08:10.523651 18019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:10.516347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60348 (local address 127.11.254.189:45185)
0812 21:08:10.516519 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:10.516523 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:10.516539 (+ 16us) server_negotiation.cc:407] Connection header received
0812 21:08:10.516558 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:10.516560 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:10.516570 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:10.516633 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:10.516932 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:10.517669 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:10.518550 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:10.518884 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:10.518947 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:10.520854 (+ 1907us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:08:10.520864 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:08:10.520864 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:08:10.520872 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:08:10.522539 (+ 1667us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:10.522837 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:10.522840 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:10.522840 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:10.522890 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:10.523046 (+ 156us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:10.523047 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:10.523047 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:10.523205 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:08:10.523275 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:10.523431 (+ 156us) server_negotiation.cc:299] Negotiation successful
0812 21:08:10.523526 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":28,"server-negotiator.queue_time_us":141,"thread_start_us":76,"threads_started":1}
I20260812 21:08:10.524288 18009 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.254.188" port: 33803 } has no permanent_uuid. Determining permanent_uuid...
I20260812 21:08:10.524626 18018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:10.524517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.188:33803 (local address 127.0.0.1:59720)
0812 21:08:10.524557 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:10.524602 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.188:33803: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260812 21:08:10.524747 17925 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.254.188:33803: connect: Connection refused (error 111)
I20260812 21:08:10.525158 18018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:10.525067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.188:33803 (local address 127.0.0.1:59734)
0812 21:08:10.525111 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:10.525135 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.188:33803: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260812 21:08:10.525246 18009 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.254.188:33803: Network error: Client connection negotiation failed: client connection to 127.11.254.188:33803: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.254.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.188.keytab.
Entry for principal kudu/127.11.254.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.188.keytab.
I20260812 21:08:10.532507 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786568607354921-12282-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786568607354921-12282-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786568607354921-12282-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786568607354921-12282-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.254.188:33803
--webserver_interface=127.11.254.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.188.keytab
--principal=kudu/127.11.254.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.254.148:41483
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.254.190:42299,127.11.254.189:45185,127.11.254.188:33803
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786568607354921-12282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786568607354921-12282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786568607354921-12282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 21:08:10.572528 17874 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.254.189" port: 45185 } attempt: 3
I20260812 21:08:10.574460 18009 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.254.188" port: 33803 } attempt: 1
I20260812 21:08:10.574903 18018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:10.574758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.188:33803 (local address 127.0.0.1:59742)
0812 21:08:10.574818 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:10.574865 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.188:33803: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260812 21:08:10.575304 18018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:10.575211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.188:33803 (local address 127.0.0.1:59752)
0812 21:08:10.575249 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:10.575282 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.188:33803: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260812 21:08:10.575417 18009 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.254.188:33803: Network error: Client connection negotiation failed: client connection to 127.11.254.188:33803: connect: Connection refused (error 111)
Aug 12 21:08:10 dist-test-slave-vpvm krb5kdc[17736](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568890, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.190@KRBTEST.COM for kudu/127.11.254.189@KRBTEST.COM
I20260812 21:08:10.579945 17880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:10.572958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.189:45185 (local address 127.0.0.1:60356)
0812 21:08:10.573006 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:10.573019 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 21:08:10.573087 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:08:10.573213 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:08:10.573214 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:08:10.573392 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:08:10.573470 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:10.573472 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:10.574201 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:10.574203 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:10.574925 (+ 722us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:10.574928 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:10.575507 (+ 579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:10.575509 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:10.575595 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:10.576068 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:08:10.576072 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:08:10.577267 (+ 1195us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:08:10.578947 (+ 1680us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:08:10.578948 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:08:10.578949 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:08:10.579136 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:08:10.579327 (+ 191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:08:10.579328 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:08:10.579328 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:08:10.579379 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:08:10.579698 (+ 319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:08:10.579699 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:08:10.579741 (+ 42us) client_negotiation.cc:769] Sending connection context
0812 21:08:10.579793 (+ 52us) client_negotiation.cc:240] Negotiation successful
0812 21:08:10.579874 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23}
I20260812 21:08:10.579998 18019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:10.573014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60356 (local address 127.11.254.189:45185)
0812 21:08:10.573088 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:10.573091 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:10.573119 (+ 28us) server_negotiation.cc:407] Connection header received
0812 21:08:10.573148 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:10.573149 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:10.573161 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:10.573203 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:10.573555 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:10.574118 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:10.575082 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:10.575429 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:10.575502 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:10.577352 (+ 1850us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:08:10.577355 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:08:10.577355 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:08:10.577364 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:08:10.578832 (+ 1468us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:10.579205 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:10.579206 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:10.579207 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:10.579271 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:10.579480 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:10.579481 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:10.579481 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:10.579639 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:08:10.579708 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:10.579828 (+ 120us) server_negotiation.cc:299] Negotiation successful
0812 21:08:10.579914 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 21:08:10.580591 17874 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.254.188" port: 33803 } has no permanent_uuid. Determining permanent_uuid...
I20260812 21:08:10.580938 17880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:10.580843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.188:33803 (local address 127.0.0.1:59764)
0812 21:08:10.580883 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:10.580912 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.188:33803: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260812 21:08:10.581218 17880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:10.581163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.188:33803 (local address 127.0.0.1:59780)
0812 21:08:10.581186 (+ 23us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:10.581205 (+ 19us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.188:33803: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":13}
W20260812 21:08:10.581291 17874 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.254.188:33803: Network error: Client connection negotiation failed: client connection to 127.11.254.188:33803: connect: Connection refused (error 111)
I20260812 21:08:10.612473 17874 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.254.188" port: 33803 } attempt: 1
I20260812 21:08:10.612986 17880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:10.612842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.188:33803 (local address 127.0.0.1:59796)
0812 21:08:10.612905 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:10.612952 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.188:33803: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260812 21:08:10.613343 17880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:10.613268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.188:33803 (local address 127.0.0.1:59806)
0812 21:08:10.613302 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:10.613326 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.188:33803: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
W20260812 21:08:10.613440 17874 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.254.188:33803: Network error: Client connection negotiation failed: client connection to 127.11.254.188:33803: connect: Connection refused (error 111)
W20260812 21:08:10.619292 18022 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 21:08:10.619511 18022 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:08:10.619529 18022 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:08:10.619547 18022 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:08:10.620690 18022 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 21:08:10.620728 18022 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:08:10.620740 18022 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:08:10.620751 18022 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 21:08:10.620761 18022 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 21:08:10.622390 18022 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.254.148:41483
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786568607354921-12282-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786568607354921-12282-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.11.254.190:42299,127.11.254.189:45185,127.11.254.188:33803
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.188.keytab
--principal=kudu/127.11.254.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.254.188:33803
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786568607354921-12282-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.11.254.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18022
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786568607354921-12282-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:08:10.622609 18022 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:08:10.622802 18022 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:08:10.627642 18027 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:08:10.627614 18038 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:08:10.627785 18022 server_base.cc:1034] running on GCE node
W20260812 21:08:10.627614 18032 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:08:10.628144 18022 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:08:10.628430 18022 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:08:10.629581 18022 hybrid_clock.cc:648] HybridClock initialized: now 1786568890629548 us; error 34 us; skew 500 ppm
Aug 12 21:08:10 dist-test-slave-vpvm krb5kdc[17736](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568890, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:08:10.632097 18022 init.cc:377] Logged in from keytab as kudu/127.11.254.188@KRBTEST.COM (short username kudu)
I20260812 21:08:10.632512 18022 webserver.cc:460] Webserver started at http://127.11.254.188:45063/ using document root <none> and password file <none>
I20260812 21:08:10.632741 18022 fs_manager.cc:359] Metadata directory not provided
I20260812 21:08:10.632791 18022 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:08:10.632921 18022 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:08:10.634034 18022 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786568607354921-12282-0/minicluster-data/master-2/data/instance:
uuid: "8277856795184162a102d80df72d4c8c"
format_stamp: "Formatted at 2026-08-12 21:08:10 on dist-test-slave-vpvm"
server_key: "35876e228ddd6732d0bce926054c61d5"
server_key_iv: "a879eba7a0a6a3291bfa2c31c91dde56"
server_key_version: "encryptionkey@0"
I20260812 21:08:10.634377 18022 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786568607354921-12282-0/minicluster-data/master-2/wal/instance:
uuid: "8277856795184162a102d80df72d4c8c"
format_stamp: "Formatted at 2026-08-12 21:08:10 on dist-test-slave-vpvm"
server_key: "35876e228ddd6732d0bce926054c61d5"
server_key_iv: "a879eba7a0a6a3291bfa2c31c91dde56"
server_key_version: "encryptionkey@0"
I20260812 21:08:10.635667 18022 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 21:08:10.636436 18051 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:08:10.636628 18022 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 21:08:10.636718 18022 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786568607354921-12282-0/minicluster-data/master-2/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786568607354921-12282-0/minicluster-data/master-2/wal
uuid: "8277856795184162a102d80df72d4c8c"
format_stamp: "Formatted at 2026-08-12 21:08:10 on dist-test-slave-vpvm"
server_key: "35876e228ddd6732d0bce926054c61d5"
server_key_iv: "a879eba7a0a6a3291bfa2c31c91dde56"
server_key_version: "encryptionkey@0"
I20260812 21:08:10.636782 18022 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786568607354921-12282-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786568607354921-12282-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786568607354921-12282-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:08:10.643550 18009 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.254.188" port: 33803 } attempt: 2
I20260812 21:08:10.644008 18018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:10.643865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.188:33803 (local address 127.0.0.1:59808)
0812 21:08:10.643925 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:10.643976 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.188:33803: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260812 21:08:10.644290 18018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:10.644219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.188:33803 (local address 127.0.0.1:59816)
0812 21:08:10.644250 (+ 31us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:10.644273 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.188:33803: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
W20260812 21:08:10.644362 18009 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.254.188:33803: Network error: Client connection negotiation failed: client connection to 127.11.254.188:33803: connect: Connection refused (error 111)
I20260812 21:08:10.648772 18022 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:08:10.650605 18022 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:08:10.650719 18022 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:08:10.654649 18022 rpc_server.cc:307] RPC server started. Bound to: 127.11.254.188:33803
I20260812 21:08:10.654692 18140 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.254.188:33803 every 8 connection(s)
I20260812 21:08:10.655035 18022 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786568607354921-12282-0/minicluster-data/master-2/data/info.pb
I20260812 21:08:10.656827 18142 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:08:10.658278 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 18022
I20260812 21:08:10.658365 12282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786568607354921-12282-0/minicluster-data/master-2/wal/instance
I20260812 21:08:10.658596 12282 external_mini_cluster.cc:1428] Setting key 1fad4408a7f74d18fa96c30c2f664bff
I20260812 21:08:10.658623 18142 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.254.190" port: 42299 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 21:08:10 dist-test-slave-vpvm krb5kdc[17736](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568890, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.188@KRBTEST.COM for kudu/127.11.254.190@KRBTEST.COM
Aug 12 21:08:10 dist-test-slave-vpvm krb5kdc[17736](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568890, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.254.188@KRBTEST.COM
I20260812 21:08:10.667203 17882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:10.659464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58538 (local address 127.11.254.190:42299)
0812 21:08:10.659508 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:10.659511 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:10.659679 (+ 168us) server_negotiation.cc:407] Connection header received
0812 21:08:10.659897 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:10.659898 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:10.659910 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:10.659958 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:10.660501 (+ 543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:10.660981 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:10.661833 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:10.662098 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:10.662158 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:10.664392 (+ 2234us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:08:10.664394 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:08:10.664395 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:08:10.664403 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:08:10.665803 (+ 1400us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:10.666250 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:10.666252 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:10.666252 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:10.666313 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:10.666571 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:10.666572 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:10.666573 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:10.666693 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:08:10.666747 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:10.667025 (+ 278us) server_negotiation.cc:299] Negotiation successful
0812 21:08:10.667109 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 21:08:10.667358 18153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:10.659411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:42299 (local address 127.0.0.1:58538)
0812 21:08:10.659577 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:10.659610 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0812 21:08:10.659826 (+ 216us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:08:10.659975 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:08:10.659978 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:08:10.660242 (+ 264us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:08:10.660408 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:10.660413 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:10.661053 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:10.661054 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:10.661750 (+ 696us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:10.661752 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:10.662172 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:10.662173 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:10.662338 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:10.662805 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:08:10.662816 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:08:10.664301 (+ 1485us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:08:10.665888 (+ 1587us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:08:10.665896 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:08:10.665898 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:08:10.666186 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:08:10.666365 (+ 179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:08:10.666365 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:08:10.666366 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:08:10.666510 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:08:10.666745 (+ 235us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:08:10.666752 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:08:10.666889 (+ 137us) client_negotiation.cc:769] Sending connection context
0812 21:08:10.666975 (+ 86us) client_negotiation.cc:240] Negotiation successful
0812 21:08:10.667141 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"mutex_wait_us":20,"thread_start_us":72,"threads_started":1}
I20260812 21:08:10.667371 18149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:10.659131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59832 (local address 127.11.254.188:33803)
0812 21:08:10.659302 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:10.659311 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:10.659343 (+ 32us) server_negotiation.cc:407] Connection header received
0812 21:08:10.659826 (+ 483us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:10.659833 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:10.659854 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:10.659941 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:10.660558 (+ 617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:10.661319 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:10.662073 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:10.662397 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:10.662454 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:10.664842 (+ 2388us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:08:10.664851 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:08:10.664851 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:08:10.664858 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:08:10.666318 (+ 1460us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:10.666634 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:10.666637 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:10.666637 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:10.666686 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:10.666847 (+ 161us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:10.666847 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:10.666848 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:10.666978 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:08:10.667111 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:10.667191 (+ 80us) server_negotiation.cc:299] Negotiation successful
0812 21:08:10.667266 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":15,"server-negotiator.queue_time_us":122,"thread_start_us":76,"threads_started":1}
I20260812 21:08:10.668397 18142 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.254.189" port: 45185 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 21:08:10 dist-test-slave-vpvm krb5kdc[17736](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568890, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.188@KRBTEST.COM for kudu/127.11.254.189@KRBTEST.COM
Aug 12 21:08:10 dist-test-slave-vpvm krb5kdc[17736](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568890, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.254.190@KRBTEST.COM
I20260812 21:08:10.674929 18019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:10.668705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60364 (local address 127.11.254.189:45185)
0812 21:08:10.668753 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:10.668755 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:10.668762 (+ 7us) server_negotiation.cc:407] Connection header received
0812 21:08:10.668785 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:10.668786 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:10.668795 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:10.668839 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:10.669123 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:10.669596 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:10.670346 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:10.670555 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:10.670609 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:10.672322 (+ 1713us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:08:10.672324 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:08:10.672324 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:08:10.672332 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:08:10.673780 (+ 1448us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:10.674217 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:10.674218 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:10.674219 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:10.674274 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:10.674435 (+ 161us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:10.674435 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:10.674436 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:10.674535 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:08:10.674593 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:10.674765 (+ 172us) server_negotiation.cc:299] Negotiation successful
0812 21:08:10.674836 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 21:08:10.674929 18153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:10.668636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.189:45185 (local address 127.0.0.1:60364)
0812 21:08:10.668675 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:10.668682 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0812 21:08:10.668710 (+ 28us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:08:10.668848 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:08:10.668848 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:08:10.668979 (+ 131us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:08:10.669053 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:10.669055 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:10.669661 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:10.669662 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:10.670273 (+ 611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:10.670275 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:10.670612 (+ 337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:10.670612 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:10.670680 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:10.671112 (+ 432us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:08:10.671117 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:08:10.672238 (+ 1121us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:08:10.673860 (+ 1622us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:08:10.673861 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:08:10.673862 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:08:10.674146 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:08:10.674323 (+ 177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:08:10.674324 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:08:10.674324 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:08:10.674384 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:08:10.674629 (+ 245us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:08:10.674630 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:08:10.674674 (+ 44us) client_negotiation.cc:769] Sending connection context
0812 21:08:10.674731 (+ 57us) client_negotiation.cc:240] Negotiation successful
0812 21:08:10.674827 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260812 21:08:10.675330 18142 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.254.188" port: 33803 } has no permanent_uuid. Determining permanent_uuid...
I20260812 21:08:10.675761 17882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:10.668568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58542 (local address 127.11.254.190:42299)
0812 21:08:10.668604 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:10.668606 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:10.668613 (+ 7us) server_negotiation.cc:407] Connection header received
0812 21:08:10.668632 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:10.668632 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:10.668641 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:10.668695 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:10.669184 (+ 489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:10.669640 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:10.670330 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:10.670531 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:10.670584 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:10.672801 (+ 2217us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:08:10.672803 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:08:10.672803 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:08:10.672810 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:08:10.674231 (+ 1421us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:10.674567 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:10.674568 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:10.674568 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:10.674617 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:10.674776 (+ 159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:10.674777 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:10.674778 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:10.674878 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:08:10.675233 (+ 355us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:10.675454 (+ 221us) server_negotiation.cc:299] Negotiation successful
0812 21:08:10.675683 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
Aug 12 21:08:10 dist-test-slave-vpvm krb5kdc[17736](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568890, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.188@KRBTEST.COM for kudu/127.11.254.188@KRBTEST.COM
Aug 12 21:08:10 dist-test-slave-vpvm krb5kdc[17736](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568890, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.254.189@KRBTEST.COM
I20260812 21:08:10.681795 18149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:10.675661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59836 (local address 127.11.254.188:33803)
0812 21:08:10.675707 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:10.675709 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:10.675912 (+ 203us) server_negotiation.cc:407] Connection header received
0812 21:08:10.676049 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:10.676050 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:10.676059 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:10.676112 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:10.676371 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:10.676897 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:10.677551 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:10.677748 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:10.677796 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:10.679410 (+ 1614us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:08:10.679411 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:08:10.679411 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:08:10.679418 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:08:10.680773 (+ 1355us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:10.681076 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:10.681077 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:10.681077 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:10.681124 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:10.681297 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:10.681298 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:10.681298 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:10.681423 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:08:10.681487 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:10.681581 (+ 94us) server_negotiation.cc:299] Negotiation successful
0812 21:08:10.681659 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260812 21:08:10.681803 18153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:10.675587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.188:33803 (local address 127.0.0.1:59836)
0812 21:08:10.675850 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:10.675862 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 21:08:10.675913 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:08:10.676121 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:08:10.676122 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:08:10.676260 (+ 138us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:08:10.676297 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:10.676298 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:10.676970 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:10.676971 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:10.677476 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:10.677478 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:10.677805 (+ 327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:10.677806 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:10.677858 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:10.678228 (+ 370us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:08:10.678232 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:08:10.679336 (+ 1104us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:08:10.680847 (+ 1511us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:08:10.680848 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:08:10.680849 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:08:10.681005 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:08:10.681186 (+ 181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:08:10.681186 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:08:10.681187 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:08:10.681244 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:08:10.681478 (+ 234us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:08:10.681478 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:08:10.681512 (+ 34us) client_negotiation.cc:769] Sending connection context
0812 21:08:10.681555 (+ 43us) client_negotiation.cc:240] Negotiation successful
0812 21:08:10.681708 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":245,"mutex_wait_us":46}
I20260812 21:08:10.683176 18019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:10.676348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60376 (local address 127.11.254.189:45185)
0812 21:08:10.676392 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:10.676395 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:10.676405 (+ 10us) server_negotiation.cc:407] Connection header received
0812 21:08:10.676424 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:10.676424 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:10.676433 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:10.676486 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:10.676870 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:10.677333 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:10.678079 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:10.678331 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:10.678379 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:10.680421 (+ 2042us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:08:10.680422 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:08:10.680422 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:08:10.680430 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:08:10.681833 (+ 1403us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:10.682437 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:10.682438 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:10.682439 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:10.682498 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:10.682690 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:10.682691 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:10.682692 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:10.682810 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:08:10.682968 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:10.683002 (+ 34us) server_negotiation.cc:299] Negotiation successful
0812 21:08:10.683077 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 21:08:10.683638 18142 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8277856795184162a102d80df72d4c8c: Bootstrap starting.
I20260812 21:08:10.684796 12282 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 21:08:10.684821 18142 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8277856795184162a102d80df72d4c8c: Neither blocks nor log segments found. Creating new log.
I20260812 21:08:10.685145 18142 log.cc:824] T 00000000000000000000000000000000 P 8277856795184162a102d80df72d4c8c: Log is configured to *not* fsync() on all Append() calls
I20260812 21:08:10.686056 18142 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8277856795184162a102d80df72d4c8c: No bootstrap required, opened a new log
I20260812 21:08:10.686159 17826 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "2d02fea4ccda4d95a55e980128dc75ea"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58542
I20260812 21:08:10.687652 18142 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8277856795184162a102d80df72d4c8c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d02fea4ccda4d95a55e980128dc75ea" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 42299 } } peers { permanent_uuid: "16f134d6ec644e2b90b113224a2816e2" member_type: VOTER last_known_addr { host: "127.11.254.189" port: 45185 } } peers { permanent_uuid: "8277856795184162a102d80df72d4c8c" member_type: VOTER last_known_addr { host: "127.11.254.188" port: 33803 } }
I20260812 21:08:10.687780 18142 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8277856795184162a102d80df72d4c8c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8277856795184162a102d80df72d4c8c, State: Initialized, Role: FOLLOWER
I20260812 21:08:10.687855 18142 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8277856795184162a102d80df72d4c8c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d02fea4ccda4d95a55e980128dc75ea" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 42299 } } peers { permanent_uuid: "16f134d6ec644e2b90b113224a2816e2" member_type: VOTER last_known_addr { host: "127.11.254.189" port: 45185 } } peers { permanent_uuid: "8277856795184162a102d80df72d4c8c" member_type: VOTER last_known_addr { host: "127.11.254.188" port: 33803 } }
I20260812 21:08:10.688166 18142 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8277856795184162a102d80df72d4c8c [sys.catalog]: configured and running, proceeding with master startup.
I20260812 21:08:10.688148 18155 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8277856795184162a102d80df72d4c8c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d02fea4ccda4d95a55e980128dc75ea" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 42299 } } peers { permanent_uuid: "16f134d6ec644e2b90b113224a2816e2" member_type: VOTER last_known_addr { host: "127.11.254.189" port: 45185 } } peers { permanent_uuid: "8277856795184162a102d80df72d4c8c" member_type: VOTER last_known_addr { host: "127.11.254.188" port: 33803 } } }
I20260812 21:08:10.688266 18155 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8277856795184162a102d80df72d4c8c [sys.catalog]: This master's current role is: FOLLOWER
I20260812 21:08:10.688587 17874 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.254.188" port: 33803 } attempt: 2
W20260812 21:08:10.690057 18178 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8277856795184162a102d80df72d4c8c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 21:08:10.690115 18178 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
Aug 12 21:08:10 dist-test-slave-vpvm krb5kdc[17736](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568890, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.190@KRBTEST.COM for kudu/127.11.254.188@KRBTEST.COM
I20260812 21:08:10.690344 18022 master_runner.cc:186] Error getting master registration for 127.11.254.190:42299: OK, instance_id { permanent_uuid: "2d02fea4ccda4d95a55e980128dc75ea" instance_seqno: 1786568890344090 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 21:08:10.693527 18022 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 21:08:10.695326 18149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:10.688908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59838 (local address 127.11.254.188:33803)
0812 21:08:10.688953 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:10.688955 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:10.688966 (+ 11us) server_negotiation.cc:407] Connection header received
0812 21:08:10.689033 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:10.689034 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:10.689044 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:10.689092 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:10.689400 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:10.690091 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:10.690822 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:10.691016 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:10.691069 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:10.692766 (+ 1697us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:08:10.692767 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:08:10.692768 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:08:10.692774 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:08:10.694127 (+ 1353us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:10.694511 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:10.694512 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:10.694513 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:10.694572 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:10.694767 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:10.694767 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:10.694768 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:10.694929 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:08:10.694993 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:10.695156 (+ 163us) server_negotiation.cc:299] Negotiation successful
0812 21:08:10.695235 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260812 21:08:10.695333 17880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:10.688864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.188:33803 (local address 127.0.0.1:59838)
0812 21:08:10.688913 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:10.688924 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 21:08:10.688964 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:08:10.689099 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:08:10.689100 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:08:10.689276 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:08:10.689328 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:10.689330 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:10.690202 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:10.690204 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:10.690734 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:10.690736 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:10.691074 (+ 338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:10.691075 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:10.691130 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:10.691539 (+ 409us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:08:10.691543 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:08:10.692689 (+ 1146us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:08:10.694204 (+ 1515us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:08:10.694205 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:08:10.694205 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:08:10.694436 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:08:10.694631 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:08:10.694632 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:08:10.694632 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:08:10.694708 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:08:10.694993 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:08:10.694993 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:08:10.695045 (+ 52us) client_negotiation.cc:769] Sending connection context
0812 21:08:10.695128 (+ 83us) client_negotiation.cc:240] Negotiation successful
0812 21:08:10.695234 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260812 21:08:10.697166 17874 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d02fea4ccda4d95a55e980128dc75ea: Bootstrap starting.
I20260812 21:08:10.697896 17874 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2d02fea4ccda4d95a55e980128dc75ea: Neither blocks nor log segments found. Creating new log.
I20260812 21:08:10.698223 17874 log.cc:824] T 00000000000000000000000000000000 P 2d02fea4ccda4d95a55e980128dc75ea: Log is configured to *not* fsync() on all Append() calls
I20260812 21:08:10.698961 17874 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d02fea4ccda4d95a55e980128dc75ea: No bootstrap required, opened a new log
I20260812 21:08:10.699975 17874 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2d02fea4ccda4d95a55e980128dc75ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d02fea4ccda4d95a55e980128dc75ea" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 42299 } } peers { permanent_uuid: "16f134d6ec644e2b90b113224a2816e2" member_type: VOTER last_known_addr { host: "127.11.254.189" port: 45185 } } peers { permanent_uuid: "8277856795184162a102d80df72d4c8c" member_type: VOTER last_known_addr { host: "127.11.254.188" port: 33803 } }
I20260812 21:08:10.700093 17874 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2d02fea4ccda4d95a55e980128dc75ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d02fea4ccda4d95a55e980128dc75ea, State: Initialized, Role: FOLLOWER
I20260812 21:08:10.700171 17874 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2d02fea4ccda4d95a55e980128dc75ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d02fea4ccda4d95a55e980128dc75ea" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 42299 } } peers { permanent_uuid: "16f134d6ec644e2b90b113224a2816e2" member_type: VOTER last_known_addr { host: "127.11.254.189" port: 45185 } } peers { permanent_uuid: "8277856795184162a102d80df72d4c8c" member_type: VOTER last_known_addr { host: "127.11.254.188" port: 33803 } }
I20260812 21:08:10.700495 17874 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2d02fea4ccda4d95a55e980128dc75ea [sys.catalog]: configured and running, proceeding with master startup.
I20260812 21:08:10.700451 18182 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d02fea4ccda4d95a55e980128dc75ea [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d02fea4ccda4d95a55e980128dc75ea" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 42299 } } peers { permanent_uuid: "16f134d6ec644e2b90b113224a2816e2" member_type: VOTER last_known_addr { host: "127.11.254.189" port: 45185 } } peers { permanent_uuid: "8277856795184162a102d80df72d4c8c" member_type: VOTER last_known_addr { host: "127.11.254.188" port: 33803 } } }
I20260812 21:08:10.700548 18182 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d02fea4ccda4d95a55e980128dc75ea [sys.catalog]: This master's current role is: FOLLOWER
I20260812 21:08:10.700618 17826 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2d02fea4ccda4d95a55e980128dc75ea [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 21:08:10.700692 17826 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2d02fea4ccda4d95a55e980128dc75ea [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:08:10.701493 17826 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2d02fea4ccda4d95a55e980128dc75ea [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d02fea4ccda4d95a55e980128dc75ea" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 42299 } } peers { permanent_uuid: "16f134d6ec644e2b90b113224a2816e2" member_type: VOTER last_known_addr { host: "127.11.254.189" port: 45185 } } peers { permanent_uuid: "8277856795184162a102d80df72d4c8c" member_type: VOTER last_known_addr { host: "127.11.254.188" port: 33803 } }
I20260812 21:08:10.701733 17826 leader_election.cc:288] T 00000000000000000000000000000000 P 2d02fea4ccda4d95a55e980128dc75ea [CANDIDATE]: Term 1 election: Requested vote from peers 16f134d6ec644e2b90b113224a2816e2 (127.11.254.189:45185), 8277856795184162a102d80df72d4c8c (127.11.254.188:33803)
I20260812 21:08:10.702446 18091 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2d02fea4ccda4d95a55e980128dc75ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "8277856795184162a102d80df72d4c8c"
I20260812 21:08:10.702602 18091 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8277856795184162a102d80df72d4c8c [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:08:10.703583 18091 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 8277856795184162a102d80df72d4c8c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2d02fea4ccda4d95a55e980128dc75ea in term 1.
W20260812 21:08:10.704021 18208 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2d02fea4ccda4d95a55e980128dc75ea: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 21:08:10.704092 18208 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 21:08:10.702986 17761 master_runner.cc:186] Error getting master registration for 127.11.254.189:45185: OK, instance_id { permanent_uuid: "16f134d6ec644e2b90b113224a2816e2" instance_seqno: 1786568890501367 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 21:08:10.705744 17761 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 21:08:10.705865 17782 leader_election.cc:302] T 00000000000000000000000000000000 P 2d02fea4ccda4d95a55e980128dc75ea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d02fea4ccda4d95a55e980128dc75ea, 8277856795184162a102d80df72d4c8c; no voters:
I20260812 21:08:10.706029 18182 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2d02fea4ccda4d95a55e980128dc75ea [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:08:10.706208 18182 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2d02fea4ccda4d95a55e980128dc75ea [term 1 LEADER]: Becoming Leader. State: Replica: 2d02fea4ccda4d95a55e980128dc75ea, State: Running, Role: LEADER
I20260812 21:08:10.706310 18182 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2d02fea4ccda4d95a55e980128dc75ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d02fea4ccda4d95a55e980128dc75ea" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 42299 } } peers { permanent_uuid: "16f134d6ec644e2b90b113224a2816e2" member_type: VOTER last_known_addr { host: "127.11.254.189" port: 45185 } } peers { permanent_uuid: "8277856795184162a102d80df72d4c8c" member_type: VOTER last_known_addr { host: "127.11.254.188" port: 33803 } }
I20260812 21:08:10.706727 18182 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d02fea4ccda4d95a55e980128dc75ea [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2d02fea4ccda4d95a55e980128dc75ea. Latest consensus state: current_term: 1 leader_uuid: "2d02fea4ccda4d95a55e980128dc75ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d02fea4ccda4d95a55e980128dc75ea" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 42299 } } peers { permanent_uuid: "16f134d6ec644e2b90b113224a2816e2" member_type: VOTER last_known_addr { host: "127.11.254.189" port: 45185 } } peers { permanent_uuid: "8277856795184162a102d80df72d4c8c" member_type: VOTER last_known_addr { host: "127.11.254.188" port: 33803 } } }
I20260812 21:08:10.706816 18182 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d02fea4ccda4d95a55e980128dc75ea [sys.catalog]: This master's current role is: LEADER
I20260812 21:08:10.712522 17882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:10.703663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58544 (local address 127.11.254.190:42299)
0812 21:08:10.703705 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:10.703708 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:10.704490 (+ 782us) server_negotiation.cc:407] Connection header received
0812 21:08:10.704657 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:10.704672 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:10.704684 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:10.704714 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:10.705491 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:10.706010 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:10.706994 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:10.707266 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:10.707309 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:10.708866 (+ 1557us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:08:10.708867 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:08:10.708868 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:08:10.708876 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:08:10.710673 (+ 1797us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:10.711339 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:10.711341 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:10.711341 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:10.711414 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:10.711739 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:10.711740 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:10.711740 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:10.711881 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:08:10.711980 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:10.712348 (+ 368us) server_negotiation.cc:299] Negotiation successful
0812 21:08:10.712423 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 21:08:10.712911 18233 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 21:08:10.713117 18233 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 21:08:10.714046 18149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:10.704492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59846 (local address 127.11.254.188:33803)
0812 21:08:10.704528 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:10.704530 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:10.706577 (+ 2047us) server_negotiation.cc:407] Connection header received
0812 21:08:10.706664 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:10.706665 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:10.706675 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:10.706702 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:10.707474 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:10.708198 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:10.709186 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:10.709482 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:10.709519 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:10.711111 (+ 1592us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:08:10.711113 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:08:10.711113 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:08:10.711121 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:08:10.712662 (+ 1541us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:10.713195 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:10.713196 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:10.713197 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:10.713255 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:10.713470 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:10.713471 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:10.713471 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:10.713597 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:08:10.713665 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:10.713857 (+ 192us) server_negotiation.cc:299] Negotiation successful
0812 21:08:10.713934 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 21:08:10.717056 18091 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8277856795184162a102d80df72d4c8c [term 1 FOLLOWER]: Refusing update from remote peer 2d02fea4ccda4d95a55e980128dc75ea: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 21:08:10.717379 18182 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 2d02fea4ccda4d95a55e980128dc75ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "8277856795184162a102d80df72d4c8c" member_type: VOTER last_known_addr { host: "127.11.254.188" port: 33803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 21:08:10.718101 18019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:10.704230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60380 (local address 127.11.254.189:45185)
0812 21:08:10.704276 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:10.704277 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:10.706927 (+ 2650us) server_negotiation.cc:407] Connection header received
0812 21:08:10.706957 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:10.706958 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:10.706970 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:10.707016 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:10.708159 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:10.708880 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:10.710214 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:10.710538 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:10.710584 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:10.714459 (+ 3875us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:08:10.714460 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:08:10.714461 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:08:10.714470 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:08:10.716469 (+ 1999us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:10.716980 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:10.716981 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:10.716981 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:10.717045 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:10.717247 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:10.717247 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:10.717248 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:10.717386 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:08:10.717500 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:10.717896 (+ 396us) server_negotiation.cc:299] Negotiation successful
0812 21:08:10.718015 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 21:08:10.718837 18155 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8277856795184162a102d80df72d4c8c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2d02fea4ccda4d95a55e980128dc75ea. Latest consensus state: current_term: 1 leader_uuid: "2d02fea4ccda4d95a55e980128dc75ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d02fea4ccda4d95a55e980128dc75ea" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 42299 } } peers { permanent_uuid: "16f134d6ec644e2b90b113224a2816e2" member_type: VOTER last_known_addr { host: "127.11.254.189" port: 45185 } } peers { permanent_uuid: "8277856795184162a102d80df72d4c8c" member_type: VOTER last_known_addr { host: "127.11.254.188" port: 33803 } } }
I20260812 21:08:10.718947 18155 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8277856795184162a102d80df72d4c8c [sys.catalog]: This master's current role is: FOLLOWER
I20260812 21:08:10.720065 18182 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d02fea4ccda4d95a55e980128dc75ea [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2d02fea4ccda4d95a55e980128dc75ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d02fea4ccda4d95a55e980128dc75ea" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 42299 } } peers { permanent_uuid: "16f134d6ec644e2b90b113224a2816e2" member_type: VOTER last_known_addr { host: "127.11.254.189" port: 45185 } } peers { permanent_uuid: "8277856795184162a102d80df72d4c8c" member_type: VOTER last_known_addr { host: "127.11.254.188" port: 33803 } } }
I20260812 21:08:10.720186 18182 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d02fea4ccda4d95a55e980128dc75ea [sys.catalog]: This master's current role is: LEADER
I20260812 21:08:10.720189 18155 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8277856795184162a102d80df72d4c8c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2d02fea4ccda4d95a55e980128dc75ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d02fea4ccda4d95a55e980128dc75ea" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 42299 } } peers { permanent_uuid: "16f134d6ec644e2b90b113224a2816e2" member_type: VOTER last_known_addr { host: "127.11.254.189" port: 45185 } } peers { permanent_uuid: "8277856795184162a102d80df72d4c8c" member_type: VOTER last_known_addr { host: "127.11.254.188" port: 33803 } } }
I20260812 21:08:10.720255 18155 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8277856795184162a102d80df72d4c8c [sys.catalog]: This master's current role is: FOLLOWER
I20260812 21:08:10.720952 18239 mvcc.cc:205] Tried to move back new op lower bound from 7317786176369438720 to 7317786176333787136. Current Snapshot: MvccSnapshot[applied={T|T < 7317786176369438720}]
I20260812 21:08:10.721076 18242 mvcc.cc:205] Tried to move back new op lower bound from 7317786176369438720 to 7317786176333787136. Current Snapshot: MvccSnapshot[applied={T|T < 7317786176369438720}]
I20260812 21:08:10.721853 18233 catalog_manager.cc:1347] Generated new cluster ID: 86ca91e15b234376a4b0091efe75f52e
I20260812 21:08:10.721906 18233 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 21:08:10.735502 18009 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.254.188" port: 33803 } attempt: 3
I20260812 21:08:10.737277 18233 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 21:08:10.738127 18233 catalog_manager.cc:1504] Loading token signing keys...
Aug 12 21:08:10 dist-test-slave-vpvm krb5kdc[17736](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568890, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.189@KRBTEST.COM for kudu/127.11.254.188@KRBTEST.COM
I20260812 21:08:10.744693 18018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:10.735831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.188:33803 (local address 127.0.0.1:59854)
0812 21:08:10.735941 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:10.735953 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 21:08:10.735992 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:08:10.736161 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:08:10.736162 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:08:10.736365 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:08:10.736424 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:10.736427 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:10.737792 (+ 1365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:10.737794 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:10.738602 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:10.738605 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:10.739398 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:10.739399 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:10.739456 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:10.740104 (+ 648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:08:10.740110 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:08:10.741618 (+ 1508us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:08:10.743570 (+ 1952us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:08:10.743571 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:08:10.743572 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:08:10.743785 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:08:10.744003 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:08:10.744004 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:08:10.744004 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:08:10.744068 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:08:10.744351 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:08:10.744352 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:08:10.744404 (+ 52us) client_negotiation.cc:769] Sending connection context
0812 21:08:10.744464 (+ 60us) client_negotiation.cc:240] Negotiation successful
0812 21:08:10.744594 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":88}
I20260812 21:08:10.745052 18233 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2d02fea4ccda4d95a55e980128dc75ea: Generated new TSK 0
I20260812 21:08:10.745319 18149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:10.735954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59854 (local address 127.11.254.188:33803)
0812 21:08:10.735996 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:10.735998 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:10.736019 (+ 21us) server_negotiation.cc:407] Connection header received
0812 21:08:10.736043 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:10.736044 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:10.736054 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:10.736102 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:10.736513 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:10.737180 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:10.738984 (+ 1804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:10.739289 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:10.739344 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:10.741716 (+ 2372us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:08:10.741717 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:08:10.741718 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:08:10.741725 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:08:10.743478 (+ 1753us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:10.743870 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:10.743872 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:10.743872 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:10.743938 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:10.744130 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:10.744131 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:10.744131 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:10.744264 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:08:10.744322 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:10.745153 (+ 831us) server_negotiation.cc:299] Negotiation successful
0812 21:08:10.745237 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 21:08:10.745452 18233 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 21:08:10.747207 18009 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 16f134d6ec644e2b90b113224a2816e2: Bootstrap starting.
I20260812 21:08:10.747968 18009 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 16f134d6ec644e2b90b113224a2816e2: Neither blocks nor log segments found. Creating new log.
I20260812 21:08:10.748281 18009 log.cc:824] T 00000000000000000000000000000000 P 16f134d6ec644e2b90b113224a2816e2: Log is configured to *not* fsync() on all Append() calls
I20260812 21:08:10.749605 18009 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 16f134d6ec644e2b90b113224a2816e2: No bootstrap required, opened a new log
I20260812 21:08:10.753410 18009 raft_consensus.cc:348] T 00000000000000000000000000000000 P 16f134d6ec644e2b90b113224a2816e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d02fea4ccda4d95a55e980128dc75ea" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 42299 } } peers { permanent_uuid: "16f134d6ec644e2b90b113224a2816e2" member_type: VOTER last_known_addr { host: "127.11.254.189" port: 45185 } } peers { permanent_uuid: "8277856795184162a102d80df72d4c8c" member_type: VOTER last_known_addr { host: "127.11.254.188" port: 33803 } }
I20260812 21:08:10.753749 18009 raft_consensus.cc:729] T 00000000000000000000000000000000 P 16f134d6ec644e2b90b113224a2816e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16f134d6ec644e2b90b113224a2816e2, State: Initialized, Role: FOLLOWER
I20260812 21:08:10.753983 18009 consensus_queue.cc:260] T 00000000000000000000000000000000 P 16f134d6ec644e2b90b113224a2816e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d02fea4ccda4d95a55e980128dc75ea" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 42299 } } peers { permanent_uuid: "16f134d6ec644e2b90b113224a2816e2" member_type: VOTER last_known_addr { host: "127.11.254.189" port: 45185 } } peers { permanent_uuid: "8277856795184162a102d80df72d4c8c" member_type: VOTER last_known_addr { host: "127.11.254.188" port: 33803 } }
I20260812 21:08:10.754421 18009 sys_catalog.cc:564] T 00000000000000000000000000000000 P 16f134d6ec644e2b90b113224a2816e2 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 21:08:10.754498 17971 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 16f134d6ec644e2b90b113224a2816e2 [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:08:10.755486 17971 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 16f134d6ec644e2b90b113224a2816e2 [term 1 FOLLOWER]: Refusing update from remote peer 2d02fea4ccda4d95a55e980128dc75ea: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 21:08:10.755697 17972 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2d02fea4ccda4d95a55e980128dc75ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "16f134d6ec644e2b90b113224a2816e2"
I20260812 21:08:10.755870 18231 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 2d02fea4ccda4d95a55e980128dc75ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "16f134d6ec644e2b90b113224a2816e2" member_type: VOTER last_known_addr { host: "127.11.254.189" port: 45185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 21:08:10.756500 17972 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 16f134d6ec644e2b90b113224a2816e2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2d02fea4ccda4d95a55e980128dc75ea in term 1.
I20260812 21:08:10.754379 18258 sys_catalog.cc:455] T 00000000000000000000000000000000 P 16f134d6ec644e2b90b113224a2816e2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d02fea4ccda4d95a55e980128dc75ea" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 42299 } } peers { permanent_uuid: "16f134d6ec644e2b90b113224a2816e2" member_type: VOTER last_known_addr { host: "127.11.254.189" port: 45185 } } peers { permanent_uuid: "8277856795184162a102d80df72d4c8c" member_type: VOTER last_known_addr { host: "127.11.254.188" port: 33803 } } }
I20260812 21:08:10.756896 18258 sys_catalog.cc:458] T 00000000000000000000000000000000 P 16f134d6ec644e2b90b113224a2816e2 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 21:08:10.758778 18274 sys_catalog.cc:455] T 00000000000000000000000000000000 P 16f134d6ec644e2b90b113224a2816e2 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2d02fea4ccda4d95a55e980128dc75ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d02fea4ccda4d95a55e980128dc75ea" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 42299 } } peers { permanent_uuid: "16f134d6ec644e2b90b113224a2816e2" member_type: VOTER last_known_addr { host: "127.11.254.189" port: 45185 } } peers { permanent_uuid: "8277856795184162a102d80df72d4c8c" member_type: VOTER last_known_addr { host: "127.11.254.188" port: 33803 } } }
I20260812 21:08:10.758864 18274 sys_catalog.cc:458] T 00000000000000000000000000000000 P 16f134d6ec644e2b90b113224a2816e2 [sys.catalog]: This master's current role is: FOLLOWER
W20260812 21:08:10.759553 18273 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 16f134d6ec644e2b90b113224a2816e2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 21:08:10.759603 18273 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 21:08:10.760922 18258 sys_catalog.cc:455] T 00000000000000000000000000000000 P 16f134d6ec644e2b90b113224a2816e2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2d02fea4ccda4d95a55e980128dc75ea. Latest consensus state: current_term: 1 leader_uuid: "2d02fea4ccda4d95a55e980128dc75ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d02fea4ccda4d95a55e980128dc75ea" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 42299 } } peers { permanent_uuid: "16f134d6ec644e2b90b113224a2816e2" member_type: VOTER last_known_addr { host: "127.11.254.189" port: 45185 } } peers { permanent_uuid: "8277856795184162a102d80df72d4c8c" member_type: VOTER last_known_addr { host: "127.11.254.188" port: 33803 } } }
I20260812 21:08:10.761053 18258 sys_catalog.cc:458] T 00000000000000000000000000000000 P 16f134d6ec644e2b90b113224a2816e2 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 21:08:10.761255 18275 mvcc.cc:205] Tried to move back new op lower bound from 7317786176485711872 to 7317786176333787136. Current Snapshot: MvccSnapshot[applied={T|T < 7317786176369438720}]
W20260812 21:08:10.781682 17882 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:08:10.779948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58554 (local address 127.11.254.190:42299)
0812 21:08:10.779998 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:10.780000 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:10.780934 (+ 934us) server_negotiation.cc:407] Connection header received
0812 21:08:10.781039 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:10.781041 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:10.781052 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:10.781083 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:10.781598 (+ 515us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58554: BlockingRecv error: recv got EOF from 127.0.0.1:58554 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260812 21:08:10.782526 18019 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:08:10.780445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60388 (local address 127.11.254.189:45185)
0812 21:08:10.780490 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:10.780492 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:10.781968 (+ 1476us) server_negotiation.cc:407] Connection header received
0812 21:08:10.782075 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:10.782076 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:10.782090 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:10.782118 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:10.782455 (+ 337us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60388: BlockingRecv error: recv got EOF from 127.0.0.1:60388 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260812 21:08:10.783308 18149 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:08:10.780798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59864 (local address 127.11.254.188:33803)
0812 21:08:10.780841 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:10.780843 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:10.782756 (+ 1913us) server_negotiation.cc:407] Connection header received
0812 21:08:10.782881 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:10.782882 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:10.782893 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:10.782920 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:10.783242 (+ 322us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59864: BlockingRecv error: recv got EOF from 127.0.0.1:59864 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260812 21:08:10.791393 18019 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:08:10.790469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60390 (local address 127.11.254.189:45185)
0812 21:08:10.790514 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:10.790516 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:10.790796 (+ 280us) server_negotiation.cc:407] Connection header received
0812 21:08:10.790912 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:10.790913 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:10.790936 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:10.790965 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:10.791332 (+ 367us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60390: BlockingRecv error: recv got EOF from 127.0.0.1:60390 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260812 21:08:10.794534 18019 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:08:10.793733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60406 (local address 127.11.254.189:45185)
0812 21:08:10.793768 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:10.793770 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:10.794028 (+ 258us) server_negotiation.cc:407] Connection header received
0812 21:08:10.794121 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:10.794122 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:10.794132 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:10.794160 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:10.794481 (+ 321us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60406: BlockingRecv error: recv got EOF from 127.0.0.1:60406 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260812 21:08:10.798638 18019 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:08:10.797775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60410 (local address 127.11.254.189:45185)
0812 21:08:10.797826 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:10.797828 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:10.798085 (+ 257us) server_negotiation.cc:407] Connection header received
0812 21:08:10.798182 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:10.798183 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:10.798195 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:10.798224 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:10.798583 (+ 359us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60410: BlockingRecv error: recv got EOF from 127.0.0.1:60410 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260812 21:08:10.804744 18019 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:08:10.803913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60422 (local address 127.11.254.189:45185)
0812 21:08:10.803953 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:10.803955 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:10.804179 (+ 224us) server_negotiation.cc:407] Connection header received
0812 21:08:10.804275 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:10.804276 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:10.804287 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:10.804314 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:10.804685 (+ 371us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60422: BlockingRecv error: recv got EOF from 127.0.0.1:60422 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260812 21:08:10.815289 18019 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:08:10.814290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60436 (local address 127.11.254.189:45185)
0812 21:08:10.814341 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:10.814343 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:10.814637 (+ 294us) server_negotiation.cc:407] Connection header received
0812 21:08:10.814751 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:10.814752 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:10.814764 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:10.814795 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:10.815224 (+ 429us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60436: BlockingRecv error: recv got EOF from 127.0.0.1:60436 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260812 21:08:10.833810 18019 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:08:10.832824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60438 (local address 127.11.254.189:45185)
0812 21:08:10.832871 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:10.832873 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:10.833156 (+ 283us) server_negotiation.cc:407] Connection header received
0812 21:08:10.833264 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:10.833265 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:10.833278 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:10.833305 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:10.833744 (+ 439us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60438: BlockingRecv error: recv got EOF from 127.0.0.1:60438 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260812 21:08:10.868232 18019 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:08:10.867338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60452 (local address 127.11.254.189:45185)
0812 21:08:10.867385 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:10.867387 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:10.867632 (+ 245us) server_negotiation.cc:407] Connection header received
0812 21:08:10.867731 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:10.867732 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:10.867745 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:10.867772 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:10.868166 (+ 394us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60452: BlockingRecv error: recv got EOF from 127.0.0.1:60452 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260812 21:08:10.934355 18019 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:08:10.933589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60460 (local address 127.11.254.189:45185)
0812 21:08:10.933659 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:10.933661 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:10.933790 (+ 129us) server_negotiation.cc:407] Connection header received
0812 21:08:10.933847 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:10.933848 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:10.933858 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:10.933910 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:10.934287 (+ 377us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60460: BlockingRecv error: recv got EOF from 127.0.0.1:60460 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
W20260812 21:08:11.064390 18019 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:08:11.063648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60468 (local address 127.11.254.189:45185)
0812 21:08:11.063722 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:11.063724 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:11.063814 (+ 90us) server_negotiation.cc:407] Connection header received
0812 21:08:11.063871 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:11.063872 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:11.063882 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:11.063936 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:11.064337 (+ 401us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60468: BlockingRecv error: recv got EOF from 127.0.0.1:60468 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
W20260812 21:08:11.322600 18019 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:08:11.321924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60472 (local address 127.11.254.189:45185)
0812 21:08:11.321997 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:11.322000 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:11.322032 (+ 32us) server_negotiation.cc:407] Connection header received
0812 21:08:11.322117 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:11.322119 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:11.322131 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:11.322183 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:11.322539 (+ 356us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60472: BlockingRecv error: recv got EOF from 127.0.0.1:60472 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
I20260812 21:08:11.348229 18366 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d02fea4ccda4d95a55e980128dc75ea [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2d02fea4ccda4d95a55e980128dc75ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d02fea4ccda4d95a55e980128dc75ea" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 42299 } } peers { permanent_uuid: "16f134d6ec644e2b90b113224a2816e2" member_type: VOTER last_known_addr { host: "127.11.254.189" port: 45185 } } peers { permanent_uuid: "8277856795184162a102d80df72d4c8c" member_type: VOTER last_known_addr { host: "127.11.254.188" port: 33803 } } }
I20260812 21:08:11.348366 18366 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d02fea4ccda4d95a55e980128dc75ea [sys.catalog]: This master's current role is: LEADER
I20260812 21:08:11.690572 18178 catalog_manager.cc:1259] Loaded cluster ID: 86ca91e15b234376a4b0091efe75f52e
I20260812 21:08:11.690658 18178 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 8277856795184162a102d80df72d4c8c: loading cluster ID for follower catalog manager: success
I20260812 21:08:11.691484 18178 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 8277856795184162a102d80df72d4c8c: acquiring CA information for follower catalog manager: success
I20260812 21:08:11.691787 18178 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 8277856795184162a102d80df72d4c8c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 21:08:11.760046 18273 catalog_manager.cc:1259] Loaded cluster ID: 86ca91e15b234376a4b0091efe75f52e
I20260812 21:08:11.760139 18273 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 16f134d6ec644e2b90b113224a2816e2: loading cluster ID for follower catalog manager: success
I20260812 21:08:11.760948 18273 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 16f134d6ec644e2b90b113224a2816e2: acquiring CA information for follower catalog manager: success
I20260812 21:08:11.761250 18273 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 16f134d6ec644e2b90b113224a2816e2: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 21:08:11.838696 18385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:11.835885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60486 (local address 127.11.254.189:45185)
0812 21:08:11.836004 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:11.836007 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:11.836092 (+ 85us) server_negotiation.cc:407] Connection header received
0812 21:08:11.836131 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:11.836132 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:11.836194 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:11.836281 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 21:08:11.836428 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:11.836941 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:11.837761 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:11.838005 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:11.838055 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:11.838340 (+ 285us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 21:08:11.838466 (+ 126us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 21:08:11.838533 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:11.838569 (+ 36us) server_negotiation.cc:299] Negotiation successful
0812 21:08:11.838593 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100,"thread_start_us":58,"threads_started":1}
I20260812 21:08:11.843209 18401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:11.840657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59878 (local address 127.11.254.188:33803)
0812 21:08:11.840760 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:11.840761 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:11.840770 (+ 9us) server_negotiation.cc:407] Connection header received
0812 21:08:11.840837 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:11.840837 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:11.840881 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:11.840964 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 21:08:11.841144 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:11.841670 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:11.842425 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:11.842651 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:11.842707 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:11.842845 (+ 138us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 21:08:11.843000 (+ 155us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 21:08:11.843077 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:11.843107 (+ 30us) server_negotiation.cc:299] Negotiation successful
0812 21:08:11.843140 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82,"thread_start_us":57,"threads_started":1}
I20260812 21:08:11.847656 18385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:11.844942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60500 (local address 127.11.254.189:45185)
0812 21:08:11.844993 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:11.844995 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:11.845121 (+ 126us) server_negotiation.cc:407] Connection header received
0812 21:08:11.845213 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:11.845214 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:11.845226 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:11.845283 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 21:08:11.845410 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:11.846195 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:11.846971 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:11.847181 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:11.847240 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:11.847365 (+ 125us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 21:08:11.847403 (+ 38us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 21:08:11.847446 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:11.847538 (+ 92us) server_negotiation.cc:299] Negotiation successful
0812 21:08:11.847570 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260812 21:08:11.848273 18413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:11.845001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58570 (local address 127.11.254.190:42299)
0812 21:08:11.845146 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:11.845148 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:11.845419 (+ 271us) server_negotiation.cc:407] Connection header received
0812 21:08:11.845475 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:11.845476 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:11.845535 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:11.845622 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 21:08:11.845711 (+ 89us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:11.846285 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:11.847012 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:11.847332 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:11.847405 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:11.847848 (+ 443us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 21:08:11.847962 (+ 114us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 21:08:11.848019 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:11.848162 (+ 143us) server_negotiation.cc:299] Negotiation successful
0812 21:08:11.848194 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":88,"threads_started":1}
I20260812 21:08:11.848729 18401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:11.845200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59888 (local address 127.11.254.188:33803)
0812 21:08:11.845238 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:11.845239 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:11.845349 (+ 110us) server_negotiation.cc:407] Connection header received
0812 21:08:11.845375 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:11.845376 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:11.845389 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:11.845436 (+ 47us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 21:08:11.845557 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:11.846288 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:11.847851 (+ 1563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:11.848113 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:11.848171 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:11.848413 (+ 242us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 21:08:11.848449 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 21:08:11.848497 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:11.848634 (+ 137us) server_negotiation.cc:299] Negotiation successful
0812 21:08:11.848664 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 21:08:11.850028 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 17761
I20260812 21:08:11.860826 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 17887
I20260812 21:08:11.874971 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 18022
2026-08-12T21:08:11Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (2972 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786568607354921-12282-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 21:08:11 dist-test-slave-vpvm krb5kdc[18421](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 21:08:11 dist-test-slave-vpvm krb5kdc[18421](info): set up 2 sockets
Aug 12 21:08:11 dist-test-slave-vpvm krb5kdc[18421](info): commencing operation
krb5kdc: starting...
W20260812 21:08:13.074133 12282 mini_kdc.cc:121] Time spent starting KDC: real 1.165s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786568607354921-12282-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786568607354921-12282-0/krb5kdc/test-user.keytab.
Aug 12 21:08:13 dist-test-slave-vpvm krb5kdc[18421](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568893, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T21:08:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T21:08:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.254.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.254.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.254.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.190.keytab.
Entry for principal kudu/127.11.254.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.190.keytab.
I20260812 21:08:13.201589 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786568607354921-12282-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786568607354921-12282-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786568607354921-12282-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.254.190:42339
--webserver_interface=127.11.254.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.190.keytab
--principal=kudu/127.11.254.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.254.148:33287
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.254.190:42339,127.11.254.189:36639,127.11.254.188:35221
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786568607354921-12282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786568607354921-12282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786568607354921-12282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 21:08:13.285287 18443 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 21:08:13.285519 18443 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:08:13.285553 18443 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:08:13.285578 18443 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:08:13.286784 18443 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 21:08:13.286825 18443 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:08:13.286842 18443 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:08:13.286859 18443 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 21:08:13.286875 18443 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 21:08:13.288388 18443 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.254.148:33287
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786568607354921-12282-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786568607354921-12282-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.254.190:42339,127.11.254.189:36639,127.11.254.188:35221
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.190.keytab
--principal=kudu/127.11.254.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.254.190:42339
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.254.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18443
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786568607354921-12282-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:08:13.288662 18443 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:08:13.288887 18443 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 21:08:13.293996 18443 server_base.cc:1034] running on GCE node
W20260812 21:08:13.293941 18449 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:08:13.293922 18452 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:08:13.293936 18450 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:08:13.294427 18443 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:08:13.294730 18443 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:08:13.295877 18443 hybrid_clock.cc:648] HybridClock initialized: now 1786568893295862 us; error 36 us; skew 500 ppm
Aug 12 21:08:13 dist-test-slave-vpvm krb5kdc[18421](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568893, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:08:13.302035 18443 init.cc:377] Logged in from keytab as kudu/127.11.254.190@KRBTEST.COM (short username kudu)
I20260812 21:08:13.302512 18443 webserver.cc:460] Webserver started at http://127.11.254.190:33015/ using document root <none> and password file <none>
I20260812 21:08:13.302721 18443 fs_manager.cc:359] Metadata directory not provided
I20260812 21:08:13.302762 18443 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:08:13.302860 18443 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:08:13.303997 18443 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786568607354921-12282-0/minicluster-data/master-0/data/instance:
uuid: "08b9af11c5584eea8c0c958f0ced907c"
format_stamp: "Formatted at 2026-08-12 21:08:13 on dist-test-slave-vpvm"
server_key: "c483f24d64cda64a9c1581ff2c32178b"
server_key_iv: "487729ec5cf1ab56f3c4c3fdd6e54d49"
server_key_version: "encryptionkey@0"
I20260812 21:08:13.304328 18443 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786568607354921-12282-0/minicluster-data/master-0/wal/instance:
uuid: "08b9af11c5584eea8c0c958f0ced907c"
format_stamp: "Formatted at 2026-08-12 21:08:13 on dist-test-slave-vpvm"
server_key: "c483f24d64cda64a9c1581ff2c32178b"
server_key_iv: "487729ec5cf1ab56f3c4c3fdd6e54d49"
server_key_version: "encryptionkey@0"
I20260812 21:08:13.305645 18443 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260812 21:08:13.306418 18462 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:08:13.306628 18443 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 21:08:13.306723 18443 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786568607354921-12282-0/minicluster-data/master-0/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786568607354921-12282-0/minicluster-data/master-0/wal
uuid: "08b9af11c5584eea8c0c958f0ced907c"
format_stamp: "Formatted at 2026-08-12 21:08:13 on dist-test-slave-vpvm"
server_key: "c483f24d64cda64a9c1581ff2c32178b"
server_key_iv: "487729ec5cf1ab56f3c4c3fdd6e54d49"
server_key_version: "encryptionkey@0"
I20260812 21:08:13.306788 18443 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786568607354921-12282-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786568607354921-12282-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786568607354921-12282-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:08:13.325062 18443 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:08:13.327543 18443 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:08:13.327674 18443 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:08:13.331821 18443 rpc_server.cc:307] RPC server started. Bound to: 127.11.254.190:42339
I20260812 21:08:13.331866 18559 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.254.190:42339 every 8 connection(s)
I20260812 21:08:13.332229 18443 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786568607354921-12282-0/minicluster-data/master-0/data/info.pb
I20260812 21:08:13.333200 18561 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:08:13.334864 18561 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.254.190" port: 42339 } has no permanent_uuid. Determining permanent_uuid...
I20260812 21:08:13.337587 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 18443
I20260812 21:08:13.337670 12282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786568607354921-12282-0/minicluster-data/master-0/wal/instance
I20260812 21:08:13.337833 12282 external_mini_cluster.cc:1428] Setting key eea9d8674ee78c60b63fabd506183da1
Aug 12 21:08:13 dist-test-slave-vpvm krb5kdc[18421](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568893, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.190@KRBTEST.COM for kudu/127.11.254.190@KRBTEST.COM
I20260812 21:08:13.345089 18563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:13.335658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35988 (local address 127.11.254.190:42339)
0812 21:08:13.335838 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:13.335848 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:13.335990 (+ 142us) server_negotiation.cc:407] Connection header received
0812 21:08:13.336543 (+ 553us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:13.336545 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:13.336569 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:13.336652 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:13.337212 (+ 560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:13.338169 (+ 957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:13.339023 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:13.339330 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:13.339443 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:13.341690 (+ 2247us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:08:13.341698 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:08:13.341698 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:08:13.341705 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:08:13.343307 (+ 1602us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:13.343776 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:13.343779 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:13.343780 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:13.343861 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:13.344108 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:13.344109 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:13.344109 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:13.344340 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:08:13.344404 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:13.344675 (+ 271us) server_negotiation.cc:299] Negotiation successful
0812 21:08:13.344886 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":114,"server-negotiator.queue_time_us":135,"thread_start_us":92,"threads_started":1}
I20260812 21:08:13.345095 18564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:13.335637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:42339 (local address 127.0.0.1:35988)
0812 21:08:13.335829 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:13.335871 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 21:08:13.336378 (+ 507us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:08:13.336670 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:08:13.336672 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:08:13.336954 (+ 282us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:08:13.337118 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:13.337122 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:13.338259 (+ 1137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:13.338260 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:13.338944 (+ 684us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:13.338947 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:13.339449 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:13.339451 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:13.339565 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:13.340074 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:08:13.340083 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:08:13.341579 (+ 1496us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:08:13.343398 (+ 1819us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:08:13.343407 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:08:13.343410 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:08:13.343691 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:08:13.343921 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:08:13.343922 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:08:13.343922 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:08:13.344046 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:08:13.344406 (+ 360us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:08:13.344412 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:08:13.344496 (+ 84us) client_negotiation.cc:769] Sending connection context
0812 21:08:13.344602 (+ 106us) client_negotiation.cc:240] Negotiation successful
0812 21:08:13.344834 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"mutex_wait_us":3,"thread_start_us":104,"threads_started":1}
I20260812 21:08:13.346139 18561 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.254.189" port: 36639 } has no permanent_uuid. Determining permanent_uuid...
I20260812 21:08:13.346518 18564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:13.346383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.189:36639 (local address 127.0.0.1:47018)
0812 21:08:13.346432 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:13.346487 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.189:36639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
WARNING: no policy specified for kudu/127.11.254.189@KRBTEST.COM; defaulting to no policy
W20260812 21:08:13.346665 18466 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.254.189:36639: connect: Connection refused (error 111)
I20260812 21:08:13.347093 18564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:13.347005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.189:36639 (local address 127.0.0.1:47022)
0812 21:08:13.347048 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:13.347075 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.189:36639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260812 21:08:13.347211 18561 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.254.189:36639: Network error: Client connection negotiation failed: client connection to 127.11.254.189:36639: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.254.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.254.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.189.keytab.
Entry for principal kudu/127.11.254.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.189.keytab.
I20260812 21:08:13.365294 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786568607354921-12282-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786568607354921-12282-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786568607354921-12282-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786568607354921-12282-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.254.189:36639
--webserver_interface=127.11.254.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.189.keytab
--principal=kudu/127.11.254.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.254.148:33287
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.254.190:42339,127.11.254.189:36639,127.11.254.188:35221
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786568607354921-12282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786568607354921-12282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786568607354921-12282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 21:08:13.396451 18561 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.254.189" port: 36639 } attempt: 1
I20260812 21:08:13.397006 18564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:13.396836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.189:36639 (local address 127.0.0.1:47034)
0812 21:08:13.396908 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:13.396963 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.189:36639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260812 21:08:13.397447 18564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:13.397356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.189:36639 (local address 127.0.0.1:47040)
0812 21:08:13.397394 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:13.397424 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.189:36639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260812 21:08:13.397588 18561 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.254.189:36639: Network error: Client connection negotiation failed: client connection to 127.11.254.189:36639: connect: Connection refused (error 111)
W20260812 21:08:13.449815 18571 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 21:08:13.450078 18571 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:08:13.450095 18571 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:08:13.450116 18571 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:08:13.451320 18571 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 21:08:13.451363 18571 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:08:13.451376 18571 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:08:13.451388 18571 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 21:08:13.451399 18571 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 21:08:13.452926 18571 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.254.148:33287
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786568607354921-12282-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786568607354921-12282-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.11.254.190:42339,127.11.254.189:36639,127.11.254.188:35221
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.189.keytab
--principal=kudu/127.11.254.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.254.189:36639
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786568607354921-12282-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.11.254.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18571
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786568607354921-12282-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:08:13.453176 18571 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:08:13.453408 18571 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 21:08:13.458393 18579 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:08:13.458415 18580 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:08:13.458389 18583 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:08:13.458629 18571 server_base.cc:1034] running on GCE node
I20260812 21:08:13.458873 18571 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:08:13.459192 18571 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:08:13.460381 18571 hybrid_clock.cc:648] HybridClock initialized: now 1786568893460365 us; error 29 us; skew 500 ppm
Aug 12 21:08:13 dist-test-slave-vpvm krb5kdc[18421](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568893, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:08:13.463219 18571 init.cc:377] Logged in from keytab as kudu/127.11.254.189@KRBTEST.COM (short username kudu)
I20260812 21:08:13.463735 18571 webserver.cc:460] Webserver started at http://127.11.254.189:41177/ using document root <none> and password file <none>
I20260812 21:08:13.463984 18571 fs_manager.cc:359] Metadata directory not provided
I20260812 21:08:13.464032 18571 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:08:13.464148 18571 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:08:13.465283 18571 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786568607354921-12282-0/minicluster-data/master-1/data/instance:
uuid: "a08c1a561f454a4cb3c7ac20e1bd179f"
format_stamp: "Formatted at 2026-08-12 21:08:13 on dist-test-slave-vpvm"
server_key: "777c38257f030da3dec7f8a69e1ceff2"
server_key_iv: "d1d682f6e35faee8367648b4189fa5fb"
server_key_version: "encryptionkey@0"
I20260812 21:08:13.465653 18571 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786568607354921-12282-0/minicluster-data/master-1/wal/instance:
uuid: "a08c1a561f454a4cb3c7ac20e1bd179f"
format_stamp: "Formatted at 2026-08-12 21:08:13 on dist-test-slave-vpvm"
server_key: "777c38257f030da3dec7f8a69e1ceff2"
server_key_iv: "d1d682f6e35faee8367648b4189fa5fb"
server_key_version: "encryptionkey@0"
I20260812 21:08:13.465787 18561 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.254.189" port: 36639 } attempt: 2
I20260812 21:08:13.466248 18564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:13.466096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.189:36639 (local address 127.0.0.1:47050)
0812 21:08:13.466163 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:13.466207 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.189:36639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260812 21:08:13.466606 18564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:13.466506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.189:36639 (local address 127.0.0.1:47064)
0812 21:08:13.466549 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:13.466580 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.189:36639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260812 21:08:13.466694 18561 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.254.189:36639: Network error: Client connection negotiation failed: client connection to 127.11.254.189:36639: connect: Connection refused (error 111)
I20260812 21:08:13.467291 18571 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 21:08:13.468079 18596 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:08:13.468233 18571 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 21:08:13.468300 18571 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786568607354921-12282-0/minicluster-data/master-1/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786568607354921-12282-0/minicluster-data/master-1/wal
uuid: "a08c1a561f454a4cb3c7ac20e1bd179f"
format_stamp: "Formatted at 2026-08-12 21:08:13 on dist-test-slave-vpvm"
server_key: "777c38257f030da3dec7f8a69e1ceff2"
server_key_iv: "d1d682f6e35faee8367648b4189fa5fb"
server_key_version: "encryptionkey@0"
I20260812 21:08:13.468359 18571 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786568607354921-12282-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786568607354921-12282-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786568607354921-12282-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:08:13.482197 18571 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:08:13.484468 18571 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:08:13.484603 18571 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:08:13.488780 18571 rpc_server.cc:307] RPC server started. Bound to: 127.11.254.189:36639
I20260812 21:08:13.488822 18688 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.254.189:36639 every 8 connection(s)
I20260812 21:08:13.489169 18571 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786568607354921-12282-0/minicluster-data/master-1/data/info.pb
I20260812 21:08:13.490217 18690 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:08:13.491258 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 18571
I20260812 21:08:13.491374 12282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786568607354921-12282-0/minicluster-data/master-1/wal/instance
I20260812 21:08:13.491608 12282 external_mini_cluster.cc:1428] Setting key 5d56120f55292789f4edd28cb436c5d8
I20260812 21:08:13.492297 18690 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.254.190" port: 42339 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 21:08:13 dist-test-slave-vpvm krb5kdc[18421](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568893, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.189@KRBTEST.COM for kudu/127.11.254.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.11.254.188@KRBTEST.COM; defaulting to no policy
I20260812 21:08:13.501785 18563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:13.493180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35998 (local address 127.11.254.190:42339)
0812 21:08:13.493251 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:13.493253 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:13.493462 (+ 209us) server_negotiation.cc:407] Connection header received
0812 21:08:13.494094 (+ 632us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:13.494096 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:13.494108 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:13.494151 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:13.494807 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:13.495308 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:13.496301 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:13.496511 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:13.496579 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:13.499061 (+ 2482us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:08:13.499063 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:08:13.499063 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:08:13.499072 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:08:13.500445 (+ 1373us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:13.500839 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:13.500839 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:13.500840 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:13.500889 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:13.501174 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:13.501175 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:13.501175 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:13.501318 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:08:13.501373 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:13.501634 (+ 261us) server_negotiation.cc:299] Negotiation successful
0812 21:08:13.501713 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260812 21:08:13.501920 18692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:13.493129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:42339 (local address 127.0.0.1:35998)
0812 21:08:13.493322 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:13.493368 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 21:08:13.493975 (+ 607us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:08:13.494226 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:08:13.494230 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:08:13.494499 (+ 269us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:08:13.494714 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:13.494719 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:13.495400 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:13.495401 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:13.496217 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:13.496220 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:13.496586 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:13.496587 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:13.496712 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:13.497288 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:08:13.497296 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:08:13.498956 (+ 1660us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:08:13.500529 (+ 1573us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:08:13.500540 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:08:13.500542 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:08:13.500774 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:08:13.500937 (+ 163us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:08:13.500938 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:08:13.500938 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:08:13.501117 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:08:13.501371 (+ 254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:08:13.501375 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:08:13.501508 (+ 133us) client_negotiation.cc:769] Sending connection context
0812 21:08:13.501590 (+ 82us) client_negotiation.cc:240] Negotiation successful
0812 21:08:13.501727 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":95,"threads_started":1}
I20260812 21:08:13.502576 18690 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.254.189" port: 36639 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.254.188@KRBTEST.COM" created.
Aug 12 21:08:13 dist-test-slave-vpvm krb5kdc[18421](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568893, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.189@KRBTEST.COM for kudu/127.11.254.189@KRBTEST.COM
I20260812 21:08:13.510608 18699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:13.502918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47076 (local address 127.11.254.189:36639)
0812 21:08:13.503091 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:13.503095 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:13.503113 (+ 18us) server_negotiation.cc:407] Connection header received
0812 21:08:13.503145 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:13.503147 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:13.503159 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:13.503220 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:13.503475 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:13.504253 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:13.505189 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:13.505537 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:13.505638 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:13.507625 (+ 1987us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:08:13.507634 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:08:13.507635 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:08:13.507641 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:08:13.509201 (+ 1560us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:13.509648 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:13.509652 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:13.509653 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:13.509718 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:13.509929 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:13.509930 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:13.509931 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:13.510183 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:08:13.510252 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:13.510417 (+ 165us) server_negotiation.cc:299] Negotiation successful
0812 21:08:13.510498 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":118,"threads_started":1}
I20260812 21:08:13.510718 18692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:13.502857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.189:36639 (local address 127.0.0.1:47076)
0812 21:08:13.502905 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:13.502916 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 21:08:13.502960 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:08:13.503229 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:08:13.503230 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:08:13.503362 (+ 132us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:08:13.503410 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:13.503412 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:13.504339 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:13.504341 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:13.505086 (+ 745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:13.505089 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:13.505646 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:13.505647 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:13.505724 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:13.506165 (+ 441us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:08:13.506170 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:08:13.507517 (+ 1347us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:08:13.509292 (+ 1775us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:08:13.509294 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:08:13.509294 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:08:13.509543 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:08:13.509793 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:08:13.509795 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:08:13.509795 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:08:13.509863 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:08:13.510257 (+ 394us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:08:13.510258 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:08:13.510314 (+ 56us) client_negotiation.cc:769] Sending connection context
0812 21:08:13.510390 (+ 76us) client_negotiation.cc:240] Negotiation successful
0812 21:08:13.510574 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27,"mutex_wait_us":75}
I20260812 21:08:13.511682 18690 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.254.188" port: 35221 } has no permanent_uuid. Determining permanent_uuid...
I20260812 21:08:13.512059 18692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:13.511938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.188:35221 (local address 127.0.0.1:42586)
0812 21:08:13.511988 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:13.512038 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.188:35221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260812 21:08:13.512198 18598 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.254.188:35221: connect: Connection refused (error 111)
I20260812 21:08:13.512710 18692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:13.512568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.188:35221 (local address 127.0.0.1:42592)
0812 21:08:13.512647 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:13.512683 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.188:35221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260812 21:08:13.512831 18690 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.254.188:35221: Network error: Client connection negotiation failed: client connection to 127.11.254.188:35221: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.254.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.188.keytab.
Entry for principal kudu/127.11.254.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.188.keytab.
I20260812 21:08:13.521622 12282 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskymWbKN/build/release/bin/kudu
/tmp/dist-test-taskymWbKN/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786568607354921-12282-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786568607354921-12282-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786568607354921-12282-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786568607354921-12282-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.254.188:35221
--webserver_interface=127.11.254.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.188.keytab
--principal=kudu/127.11.254.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.254.148:33287
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.254.190:42339,127.11.254.189:36639,127.11.254.188:35221
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786568607354921-12282-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786568607354921-12282-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786568607354921-12282-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 21:08:13.557876 18561 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.254.189" port: 36639 } attempt: 3
I20260812 21:08:13.562054 18690 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.254.188" port: 35221 } attempt: 1
I20260812 21:08:13.562510 18692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:13.562360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.188:35221 (local address 127.0.0.1:42594)
0812 21:08:13.562425 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:13.562468 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.188:35221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
Aug 12 21:08:13 dist-test-slave-vpvm krb5kdc[18421](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568893, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.190@KRBTEST.COM for kudu/127.11.254.189@KRBTEST.COM
I20260812 21:08:13.562839 18692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:13.562753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.188:35221 (local address 127.0.0.1:42604)
0812 21:08:13.562786 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:13.562817 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.188:35221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
W20260812 21:08:13.562929 18690 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.254.188:35221: Network error: Client connection negotiation failed: client connection to 127.11.254.188:35221: connect: Connection refused (error 111)
I20260812 21:08:13.565665 18564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:13.558349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.189:36639 (local address 127.0.0.1:47084)
0812 21:08:13.558424 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:13.558440 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 21:08:13.558508 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:08:13.558652 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:08:13.558653 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:08:13.558862 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:08:13.558944 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:13.558947 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:13.559681 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:13.559682 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:13.560494 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:13.560498 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:13.560904 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:13.560905 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:13.560986 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:13.561453 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:08:13.561459 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:08:13.563013 (+ 1554us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:08:13.564689 (+ 1676us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:08:13.564690 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:08:13.564690 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:08:13.564879 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:08:13.565076 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:08:13.565077 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:08:13.565077 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:08:13.565129 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:08:13.565375 (+ 246us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:08:13.565375 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:08:13.565419 (+ 44us) client_negotiation.cc:769] Sending connection context
0812 21:08:13.565506 (+ 87us) client_negotiation.cc:240] Negotiation successful
0812 21:08:13.565595 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260812 21:08:13.565717 18699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:13.558443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47084 (local address 127.11.254.189:36639)
0812 21:08:13.558494 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:13.558496 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:13.558510 (+ 14us) server_negotiation.cc:407] Connection header received
0812 21:08:13.558567 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:13.558568 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:13.558579 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:13.558634 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:13.559053 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:13.559599 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:13.560596 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:13.560822 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:13.560879 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:13.563103 (+ 2224us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:08:13.563104 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:08:13.563105 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:08:13.563112 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:08:13.564594 (+ 1482us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:13.564952 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:13.564953 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:13.564953 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:13.565017 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:13.565179 (+ 162us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:13.565180 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:13.565180 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:13.565319 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:08:13.565379 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:13.565540 (+ 161us) server_negotiation.cc:299] Negotiation successful
0812 21:08:13.565632 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 21:08:13.566062 18561 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.254.188" port: 35221 } has no permanent_uuid. Determining permanent_uuid...
I20260812 21:08:13.566447 18564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:13.566321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.188:35221 (local address 127.0.0.1:42620)
0812 21:08:13.566372 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:13.566416 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.188:35221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260812 21:08:13.566810 18564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:13.566735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.188:35221 (local address 127.0.0.1:42636)
0812 21:08:13.566768 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:13.566792 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.188:35221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260812 21:08:13.566890 18561 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.254.188:35221: Network error: Client connection negotiation failed: client connection to 127.11.254.188:35221: connect: Connection refused (error 111)
I20260812 21:08:13.598045 18561 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.254.188" port: 35221 } attempt: 1
I20260812 21:08:13.598591 18564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:13.598389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.188:35221 (local address 127.0.0.1:42646)
0812 21:08:13.598501 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:13.598553 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.188:35221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260812 21:08:13.599005 18564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:13.598882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.188:35221 (local address 127.0.0.1:42660)
0812 21:08:13.598950 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:13.598976 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.188:35221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260812 21:08:13.599110 18561 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.254.188:35221: Network error: Client connection negotiation failed: client connection to 127.11.254.188:35221: connect: Connection refused (error 111)
W20260812 21:08:13.611565 18705 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 21:08:13.611819 18705 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 21:08:13.611845 18705 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 21:08:13.611876 18705 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 21:08:13.613181 18705 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 21:08:13.613240 18705 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 21:08:13.613260 18705 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 21:08:13.613277 18705 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 21:08:13.613292 18705 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 21:08:13.614992 18705 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.254.148:33287
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786568607354921-12282-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786568607354921-12282-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.11.254.190:42339,127.11.254.189:36639,127.11.254.188:35221
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786568607354921-12282-0/krb5kdc/kudu_127.11.254.188.keytab
--principal=kudu/127.11.254.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.254.188:35221
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786568607354921-12282-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.11.254.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18705
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786568607354921-12282-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 20:07:25 UTC on ccycloud.quasar-nfsqdf.root.comops.site
build id 4583594
I20260812 21:08:13.615272 18705 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 21:08:13.615504 18705 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 21:08:13.620694 18705 server_base.cc:1034] running on GCE node
W20260812 21:08:13.620675 18716 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:08:13.620688 18712 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 21:08:13.620675 18713 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 21:08:13.621088 18705 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 21:08:13.621389 18705 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 21:08:13.622543 18705 hybrid_clock.cc:648] HybridClock initialized: now 1786568893622526 us; error 33 us; skew 500 ppm
Aug 12 21:08:13 dist-test-slave-vpvm krb5kdc[18421](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568893, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 21:08:13.625077 18705 init.cc:377] Logged in from keytab as kudu/127.11.254.188@KRBTEST.COM (short username kudu)
I20260812 21:08:13.625524 18705 webserver.cc:460] Webserver started at http://127.11.254.188:36483/ using document root <none> and password file <none>
I20260812 21:08:13.625743 18705 fs_manager.cc:359] Metadata directory not provided
I20260812 21:08:13.625789 18705 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 21:08:13.625906 18705 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 21:08:13.627053 18705 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786568607354921-12282-0/minicluster-data/master-2/data/instance:
uuid: "fe908e6232244ca8b51f8799f520e72b"
format_stamp: "Formatted at 2026-08-12 21:08:13 on dist-test-slave-vpvm"
server_key: "b37fb9c9a1c3958cca935a7dd031d978"
server_key_iv: "5965909cbe49375b0cb51d4094c6972c"
server_key_version: "encryptionkey@0"
I20260812 21:08:13.627398 18705 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786568607354921-12282-0/minicluster-data/master-2/wal/instance:
uuid: "fe908e6232244ca8b51f8799f520e72b"
format_stamp: "Formatted at 2026-08-12 21:08:13 on dist-test-slave-vpvm"
server_key: "b37fb9c9a1c3958cca935a7dd031d978"
server_key_iv: "5965909cbe49375b0cb51d4094c6972c"
server_key_version: "encryptionkey@0"
I20260812 21:08:13.628757 18705 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 21:08:13.629455 18731 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 21:08:13.629668 18705 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 21:08:13.629757 18705 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786568607354921-12282-0/minicluster-data/master-2/data,/tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786568607354921-12282-0/minicluster-data/master-2/wal
uuid: "fe908e6232244ca8b51f8799f520e72b"
format_stamp: "Formatted at 2026-08-12 21:08:13 on dist-test-slave-vpvm"
server_key: "b37fb9c9a1c3958cca935a7dd031d978"
server_key_iv: "5965909cbe49375b0cb51d4094c6972c"
server_key_version: "encryptionkey@0"
I20260812 21:08:13.629824 18705 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786568607354921-12282-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786568607354921-12282-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786568607354921-12282-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 21:08:13.631095 18690 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.254.188" port: 35221 } attempt: 2
I20260812 21:08:13.631505 18692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:13.631380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.188:35221 (local address 127.0.0.1:42664)
0812 21:08:13.631434 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:13.631469 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.188:35221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260812 21:08:13.631812 18692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:13.631730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.188:35221 (local address 127.0.0.1:42668)
0812 21:08:13.631764 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:13.631789 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.254.188:35221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
W20260812 21:08:13.631898 18690 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.254.188:35221: Network error: Client connection negotiation failed: client connection to 127.11.254.188:35221: connect: Connection refused (error 111)
I20260812 21:08:13.646828 18705 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 21:08:13.649087 18705 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 21:08:13.649248 18705 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 21:08:13.653748 18705 rpc_server.cc:307] RPC server started. Bound to: 127.11.254.188:35221
I20260812 21:08:13.653828 18830 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.254.188:35221 every 8 connection(s)
I20260812 21:08:13.654206 18705 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786568607354921-12282-0/minicluster-data/master-2/data/info.pb
I20260812 21:08:13.655233 18831 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 21:08:13.656927 18831 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.254.190" port: 42339 } has no permanent_uuid. Determining permanent_uuid...
I20260812 21:08:13.657922 12282 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskymWbKN/build/release/bin/kudu as pid 18705
I20260812 21:08:13.658099 12282 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskymWbKN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786568607354921-12282-0/minicluster-data/master-2/wal/instance
I20260812 21:08:13.658339 12282 external_mini_cluster.cc:1428] Setting key 995593e38be9bfa6e0b97057fa1bf352
Aug 12 21:08:13 dist-test-slave-vpvm krb5kdc[18421](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568893, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.188@KRBTEST.COM for kudu/127.11.254.190@KRBTEST.COM
Aug 12 21:08:13 dist-test-slave-vpvm krb5kdc[18421](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568893, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.254.188@KRBTEST.COM
I20260812 21:08:13.667412 18563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:13.657865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36014 (local address 127.11.254.190:42339)
0812 21:08:13.657925 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:13.657927 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:13.658115 (+ 188us) server_negotiation.cc:407] Connection header received
0812 21:08:13.658814 (+ 699us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:13.658816 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:13.658829 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:13.658881 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:13.659535 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:13.660110 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:13.661423 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:13.661673 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:13.661748 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:13.664450 (+ 2702us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:08:13.664452 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:08:13.664452 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:08:13.664460 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:08:13.665797 (+ 1337us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:13.666269 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:13.666271 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:13.666271 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:13.666338 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:13.666678 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:13.666680 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:13.666680 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:13.666835 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:08:13.666905 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:13.667220 (+ 315us) server_negotiation.cc:299] Negotiation successful
0812 21:08:13.667312 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260812 21:08:13.667531 18833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:13.657702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.190:42339 (local address 127.0.0.1:36014)
0812 21:08:13.657922 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:13.658022 (+ 100us) client_negotiation.cc:174] Beginning negotiation
0812 21:08:13.658705 (+ 683us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:08:13.658957 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:08:13.658960 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:08:13.659223 (+ 263us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:08:13.659430 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:13.659434 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:13.660198 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:13.660199 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:13.661335 (+ 1136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:13.661337 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:13.661758 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:13.661759 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:13.661872 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:13.662444 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:08:13.662455 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:08:13.664339 (+ 1884us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:08:13.665886 (+ 1547us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:08:13.665894 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:08:13.665897 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:08:13.666195 (+ 298us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:08:13.666430 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:08:13.666431 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:08:13.666431 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:08:13.666596 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:08:13.666916 (+ 320us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:08:13.666923 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:08:13.667066 (+ 143us) client_negotiation.cc:769] Sending connection context
0812 21:08:13.667157 (+ 91us) client_negotiation.cc:240] Negotiation successful
0812 21:08:13.667317 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":104,"threads_started":1}
I20260812 21:08:13.668203 18831 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.254.189" port: 36639 } has no permanent_uuid. Determining permanent_uuid...
I20260812 21:08:13.668821 18835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:13.658775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42678 (local address 127.11.254.188:35221)
0812 21:08:13.659589 (+ 814us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:13.659593 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:13.659609 (+ 16us) server_negotiation.cc:407] Connection header received
0812 21:08:13.659628 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:13.659630 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:13.659648 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:13.659715 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:13.660354 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:13.661476 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:13.662462 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:13.662889 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:13.662962 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:13.665532 (+ 2570us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:08:13.665543 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:08:13.665543 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:08:13.665550 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:08:13.667382 (+ 1832us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:13.667798 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:13.667801 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:13.667802 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:13.667854 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:13.668021 (+ 167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:13.668021 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:13.668022 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:13.668185 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:08:13.668323 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:13.668629 (+ 306us) server_negotiation.cc:299] Negotiation successful
0812 21:08:13.668716 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":786,"thread_start_us":90,"threads_started":1}
Aug 12 21:08:13 dist-test-slave-vpvm krb5kdc[18421](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568893, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.188@KRBTEST.COM for kudu/127.11.254.189@KRBTEST.COM
Aug 12 21:08:13 dist-test-slave-vpvm krb5kdc[18421](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568893, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.254.190@KRBTEST.COM
I20260812 21:08:13.674360 18561 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.254.188" port: 35221 } attempt: 2
I20260812 21:08:13.675706 18833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:13.668478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.189:36639 (local address 127.0.0.1:47092)
0812 21:08:13.668523 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:13.668533 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 21:08:13.668564 (+ 31us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:08:13.668760 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:08:13.668761 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:08:13.668901 (+ 140us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:08:13.668965 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:13.668967 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:13.669853 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:13.669855 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:13.670448 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:13.670450 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:13.670838 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:13.670839 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:13.670913 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:13.671370 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:08:13.671375 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:08:13.672698 (+ 1323us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:08:13.674358 (+ 1660us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:08:13.674359 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:08:13.674359 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:08:13.674589 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:08:13.675041 (+ 452us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:08:13.675042 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:08:13.675043 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:08:13.675121 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:08:13.675428 (+ 307us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:08:13.675428 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:08:13.675478 (+ 50us) client_negotiation.cc:769] Sending connection context
0812 21:08:13.675544 (+ 66us) client_negotiation.cc:240] Negotiation successful
0812 21:08:13.675639 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260812 21:08:13.675746 18699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:13.668618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47092 (local address 127.11.254.189:36639)
0812 21:08:13.668666 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:13.668668 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:13.668674 (+ 6us) server_negotiation.cc:407] Connection header received
0812 21:08:13.668694 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:13.668695 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:13.668703 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:13.668746 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:13.669065 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:13.669771 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:13.670527 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:13.670773 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:13.670834 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:13.672785 (+ 1951us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:08:13.672787 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:08:13.672788 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:08:13.672797 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:08:13.674203 (+ 1406us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:13.674898 (+ 695us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:13.674900 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:13.674900 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:13.674965 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:13.675193 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:13.675195 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:13.675195 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:13.675350 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:08:13.675436 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:13.675580 (+ 144us) server_negotiation.cc:299] Negotiation successful
0812 21:08:13.675662 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 21:08:13.676060 18831 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.254.188" port: 35221 } has no permanent_uuid. Determining permanent_uuid...
I20260812 21:08:13.677373 18563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:13.669583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36018 (local address 127.11.254.190:42339)
0812 21:08:13.669626 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:13.669628 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:13.669711 (+ 83us) server_negotiation.cc:407] Connection header received
0812 21:08:13.669770 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:13.669771 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:13.669780 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:13.669825 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:13.670327 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:13.670781 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:13.671619 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:13.671827 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:13.671887 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:13.674422 (+ 2535us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:08:13.674424 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:08:13.674424 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:08:13.674432 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:08:13.676026 (+ 1594us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:13.676393 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:13.676394 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:13.676395 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:13.676467 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:13.676661 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:13.676662 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:13.676662 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:13.676784 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:08:13.676946 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:13.677163 (+ 217us) server_negotiation.cc:299] Negotiation successful
0812 21:08:13.677280 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":30,"server-negotiator.queue_time_us":24}
Aug 12 21:08:13 dist-test-slave-vpvm krb5kdc[18421](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568893, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.190@KRBTEST.COM for kudu/127.11.254.188@KRBTEST.COM
Aug 12 21:08:13 dist-test-slave-vpvm krb5kdc[18421](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568893, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.188@KRBTEST.COM for kudu/127.11.254.188@KRBTEST.COM
Aug 12 21:08:13 dist-test-slave-vpvm krb5kdc[18421](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568893, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.254.189@KRBTEST.COM
I20260812 21:08:13.683424 18835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:13.674849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42680 (local address 127.11.254.188:35221)
0812 21:08:13.674890 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:13.674891 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:13.674908 (+ 17us) server_negotiation.cc:407] Connection header received
0812 21:08:13.674928 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:13.674929 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:13.674940 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:13.674996 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:13.676052 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:13.677005 (+ 953us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:13.677930 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:13.678258 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:13.678560 (+ 302us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:13.680176 (+ 1616us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:08:13.680177 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:08:13.680177 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:08:13.680184 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:08:13.681697 (+ 1513us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:13.682091 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:13.682092 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:13.682093 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:13.682140 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:13.682671 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:13.682672 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:13.682673 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:13.682807 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:08:13.682869 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:13.683202 (+ 333us) server_negotiation.cc:299] Negotiation successful
0812 21:08:13.683333 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 21:08:13.683583 18564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:13.674663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.188:35221 (local address 127.0.0.1:42680)
0812 21:08:13.674711 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:13.674722 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 21:08:13.674759 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:08:13.675017 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:08:13.675018 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:08:13.675436 (+ 418us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:08:13.675484 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:13.675487 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:13.677090 (+ 1603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:13.677091 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:13.677841 (+ 750us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:13.677844 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:13.678349 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:13.678350 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:13.678414 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:13.678809 (+ 395us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:08:13.678814 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:08:13.680097 (+ 1283us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:08:13.681769 (+ 1672us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:08:13.681770 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:08:13.681770 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:08:13.682018 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:08:13.682192 (+ 174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:08:13.682193 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:08:13.682193 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:08:13.682252 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:08:13.683040 (+ 788us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:08:13.683041 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:08:13.683084 (+ 43us) client_negotiation.cc:769] Sending connection context
0812 21:08:13.683161 (+ 77us) client_negotiation.cc:240] Negotiation successful
0812 21:08:13.683264 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26}
I20260812 21:08:13.684537 18833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:13.676320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.188:35221 (local address 127.0.0.1:42686)
0812 21:08:13.676362 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:13.676371 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 21:08:13.676401 (+ 30us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:08:13.676624 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:08:13.676625 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:08:13.676749 (+ 124us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:08:13.676776 (+ 27us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:13.676778 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:13.677508 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:13.677509 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:13.678326 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:13.678328 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:13.678678 (+ 350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:13.678679 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:13.678758 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:13.679062 (+ 304us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:08:13.679067 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:08:13.680651 (+ 1584us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:08:13.683529 (+ 2878us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:08:13.683529 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:08:13.683530 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:08:13.683709 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:08:13.683879 (+ 170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:08:13.683880 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:08:13.683880 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:08:13.683930 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:08:13.684217 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:08:13.684218 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:08:13.684258 (+ 40us) client_negotiation.cc:769] Sending connection context
0812 21:08:13.684311 (+ 53us) client_negotiation.cc:240] Negotiation successful
0812 21:08:13.684414 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260812 21:08:13.684631 18836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:13.676357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42686 (local address 127.11.254.188:35221)
0812 21:08:13.676512 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:13.676513 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:13.676521 (+ 8us) server_negotiation.cc:407] Connection header received
0812 21:08:13.676538 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:13.676538 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:13.676546 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:13.676610 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:13.676833 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:13.677428 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:13.678388 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:13.678619 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:13.678680 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:13.680751 (+ 2071us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:08:13.680752 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:08:13.680752 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:08:13.680758 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:08:13.683447 (+ 2689us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:13.683769 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:13.683770 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:13.683770 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:13.683813 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:13.683982 (+ 169us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:13.683983 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:13.683983 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:13.684156 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:08:13.684216 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:13.684351 (+ 135us) server_negotiation.cc:299] Negotiation successful
0812 21:08:13.684468 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":46,"server-negotiator.queue_time_us":135,"thread_start_us":93,"threads_started":1}
I20260812 21:08:13.685834 18699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:13.678032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47096 (local address 127.11.254.189:36639)
0812 21:08:13.678066 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:13.678068 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:13.678075 (+ 7us) server_negotiation.cc:407] Connection header received
0812 21:08:13.678102 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:13.678103 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:13.678111 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:13.678163 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:13.678754 (+ 591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:13.679312 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:13.679984 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:13.680208 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:13.680270 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:13.682662 (+ 2392us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:08:13.682664 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:08:13.682664 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:08:13.682671 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:08:13.684465 (+ 1794us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:13.684835 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:13.684836 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:13.684836 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:13.684891 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:13.685109 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:13.685110 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:13.685111 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:13.685237 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:08:13.685410 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:13.685662 (+ 252us) server_negotiation.cc:299] Negotiation successful
0812 21:08:13.685745 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260812 21:08:13.685933 18561 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 08b9af11c5584eea8c0c958f0ced907c: Bootstrap starting.
I20260812 21:08:13.686108 18831 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fe908e6232244ca8b51f8799f520e72b: Bootstrap starting.
I20260812 21:08:13.686720 18561 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 08b9af11c5584eea8c0c958f0ced907c: Neither blocks nor log segments found. Creating new log.
I20260812 21:08:13.686810 18831 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fe908e6232244ca8b51f8799f520e72b: Neither blocks nor log segments found. Creating new log.
I20260812 21:08:13.687029 18561 log.cc:824] T 00000000000000000000000000000000 P 08b9af11c5584eea8c0c958f0ced907c: Log is configured to *not* fsync() on all Append() calls
I20260812 21:08:13.687153 18831 log.cc:824] T 00000000000000000000000000000000 P fe908e6232244ca8b51f8799f520e72b: Log is configured to *not* fsync() on all Append() calls
I20260812 21:08:13.687662 12282 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 21:08:13.687889 18561 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 08b9af11c5584eea8c0c958f0ced907c: No bootstrap required, opened a new log
I20260812 21:08:13.687889 18831 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fe908e6232244ca8b51f8799f520e72b: No bootstrap required, opened a new log
I20260812 21:08:13.688417 18510 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "08b9af11c5584eea8c0c958f0ced907c"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36018
I20260812 21:08:13.689889 18561 raft_consensus.cc:348] T 00000000000000000000000000000000 P 08b9af11c5584eea8c0c958f0ced907c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08b9af11c5584eea8c0c958f0ced907c" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 42339 } } peers { permanent_uuid: "a08c1a561f454a4cb3c7ac20e1bd179f" member_type: VOTER last_known_addr { host: "127.11.254.189" port: 36639 } } peers { permanent_uuid: "fe908e6232244ca8b51f8799f520e72b" member_type: VOTER last_known_addr { host: "127.11.254.188" port: 35221 } }
I20260812 21:08:13.689891 18831 raft_consensus.cc:348] T 00000000000000000000000000000000 P fe908e6232244ca8b51f8799f520e72b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08b9af11c5584eea8c0c958f0ced907c" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 42339 } } peers { permanent_uuid: "a08c1a561f454a4cb3c7ac20e1bd179f" member_type: VOTER last_known_addr { host: "127.11.254.189" port: 36639 } } peers { permanent_uuid: "fe908e6232244ca8b51f8799f520e72b" member_type: VOTER last_known_addr { host: "127.11.254.188" port: 35221 } }
I20260812 21:08:13.690058 18831 raft_consensus.cc:729] T 00000000000000000000000000000000 P fe908e6232244ca8b51f8799f520e72b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fe908e6232244ca8b51f8799f520e72b, State: Initialized, Role: FOLLOWER
I20260812 21:08:13.690058 18561 raft_consensus.cc:729] T 00000000000000000000000000000000 P 08b9af11c5584eea8c0c958f0ced907c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08b9af11c5584eea8c0c958f0ced907c, State: Initialized, Role: FOLLOWER
I20260812 21:08:13.690141 18831 consensus_queue.cc:260] T 00000000000000000000000000000000 P fe908e6232244ca8b51f8799f520e72b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08b9af11c5584eea8c0c958f0ced907c" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 42339 } } peers { permanent_uuid: "a08c1a561f454a4cb3c7ac20e1bd179f" member_type: VOTER last_known_addr { host: "127.11.254.189" port: 36639 } } peers { permanent_uuid: "fe908e6232244ca8b51f8799f520e72b" member_type: VOTER last_known_addr { host: "127.11.254.188" port: 35221 } }
I20260812 21:08:13.690141 18561 consensus_queue.cc:260] T 00000000000000000000000000000000 P 08b9af11c5584eea8c0c958f0ced907c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08b9af11c5584eea8c0c958f0ced907c" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 42339 } } peers { permanent_uuid: "a08c1a561f454a4cb3c7ac20e1bd179f" member_type: VOTER last_known_addr { host: "127.11.254.189" port: 36639 } } peers { permanent_uuid: "fe908e6232244ca8b51f8799f520e72b" member_type: VOTER last_known_addr { host: "127.11.254.188" port: 35221 } }
I20260812 21:08:13.690455 18831 sys_catalog.cc:564] T 00000000000000000000000000000000 P fe908e6232244ca8b51f8799f520e72b [sys.catalog]: configured and running, proceeding with master startup.
I20260812 21:08:13.690455 18561 sys_catalog.cc:564] T 00000000000000000000000000000000 P 08b9af11c5584eea8c0c958f0ced907c [sys.catalog]: configured and running, proceeding with master startup.
I20260812 21:08:13.690500 18510 raft_consensus.cc:482] T 00000000000000000000000000000000 P 08b9af11c5584eea8c0c958f0ced907c [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 21:08:13.690508 18847 sys_catalog.cc:455] T 00000000000000000000000000000000 P fe908e6232244ca8b51f8799f520e72b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08b9af11c5584eea8c0c958f0ced907c" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 42339 } } peers { permanent_uuid: "a08c1a561f454a4cb3c7ac20e1bd179f" member_type: VOTER last_known_addr { host: "127.11.254.189" port: 36639 } } peers { permanent_uuid: "fe908e6232244ca8b51f8799f520e72b" member_type: VOTER last_known_addr { host: "127.11.254.188" port: 35221 } } }
I20260812 21:08:13.690575 18510 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 08b9af11c5584eea8c0c958f0ced907c [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:08:13.690606 18847 sys_catalog.cc:458] T 00000000000000000000000000000000 P fe908e6232244ca8b51f8799f520e72b [sys.catalog]: This master's current role is: FOLLOWER
I20260812 21:08:13.691402 18510 raft_consensus.cc:504] T 00000000000000000000000000000000 P 08b9af11c5584eea8c0c958f0ced907c [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08b9af11c5584eea8c0c958f0ced907c" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 42339 } } peers { permanent_uuid: "a08c1a561f454a4cb3c7ac20e1bd179f" member_type: VOTER last_known_addr { host: "127.11.254.189" port: 36639 } } peers { permanent_uuid: "fe908e6232244ca8b51f8799f520e72b" member_type: VOTER last_known_addr { host: "127.11.254.188" port: 35221 } }
I20260812 21:08:13.691677 18510 leader_election.cc:288] T 00000000000000000000000000000000 P 08b9af11c5584eea8c0c958f0ced907c [CANDIDATE]: Term 1 election: Requested vote from peers a08c1a561f454a4cb3c7ac20e1bd179f (127.11.254.189:36639), fe908e6232244ca8b51f8799f520e72b (127.11.254.188:35221)
W20260812 21:08:13.692328 18886 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 08b9af11c5584eea8c0c958f0ced907c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 21:08:13.693117 18886 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 21:08:13.693501 18775 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "08b9af11c5584eea8c0c958f0ced907c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "fe908e6232244ca8b51f8799f520e72b"
I20260812 21:08:13.693624 18775 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fe908e6232244ca8b51f8799f520e72b [term 0 FOLLOWER]: Advancing to term 1
W20260812 21:08:13.692868 18888 catalog_manager.cc:1558] T 00000000000000000000000000000000 P fe908e6232244ca8b51f8799f520e72b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260812 21:08:13.694608 18775 raft_consensus.cc:2454] T 00000000000000000000000000000000 P fe908e6232244ca8b51f8799f520e72b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 08b9af11c5584eea8c0c958f0ced907c in term 1.
I20260812 21:08:13.695050 18705 master_runner.cc:186] Error getting master registration for 127.11.254.189:36639: OK, instance_id { permanent_uuid: "a08c1a561f454a4cb3c7ac20e1bd179f" instance_seqno: 1786568893487590 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 21:08:13.697250 18705 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 21:08:13.695461 18443 master_runner.cc:186] Error getting master registration for 127.11.254.189:36639: OK, instance_id { permanent_uuid: "a08c1a561f454a4cb3c7ac20e1bd179f" instance_seqno: 1786568893487590 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 21:08:13.698058 18463 leader_election.cc:302] T 00000000000000000000000000000000 P 08b9af11c5584eea8c0c958f0ced907c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08b9af11c5584eea8c0c958f0ced907c, fe908e6232244ca8b51f8799f520e72b; no voters:
I20260812 21:08:13.697918 18443 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260812 21:08:13.694615 18888 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 21:08:13.698330 18903 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 08b9af11c5584eea8c0c958f0ced907c [term 1 FOLLOWER]: Leader election won for term 1
I20260812 21:08:13.690493 18850 sys_catalog.cc:455] T 00000000000000000000000000000000 P 08b9af11c5584eea8c0c958f0ced907c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08b9af11c5584eea8c0c958f0ced907c" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 42339 } } peers { permanent_uuid: "a08c1a561f454a4cb3c7ac20e1bd179f" member_type: VOTER last_known_addr { host: "127.11.254.189" port: 36639 } } peers { permanent_uuid: "fe908e6232244ca8b51f8799f520e72b" member_type: VOTER last_known_addr { host: "127.11.254.188" port: 35221 } } }
I20260812 21:08:13.698676 18850 sys_catalog.cc:458] T 00000000000000000000000000000000 P 08b9af11c5584eea8c0c958f0ced907c [sys.catalog]: This master's current role is: FOLLOWER
I20260812 21:08:13.698918 18903 raft_consensus.cc:686] T 00000000000000000000000000000000 P 08b9af11c5584eea8c0c958f0ced907c [term 1 LEADER]: Becoming Leader. State: Replica: 08b9af11c5584eea8c0c958f0ced907c, State: Running, Role: LEADER
I20260812 21:08:13.699172 18903 consensus_queue.cc:237] T 00000000000000000000000000000000 P 08b9af11c5584eea8c0c958f0ced907c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08b9af11c5584eea8c0c958f0ced907c" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 42339 } } peers { permanent_uuid: "a08c1a561f454a4cb3c7ac20e1bd179f" member_type: VOTER last_known_addr { host: "127.11.254.189" port: 36639 } } peers { permanent_uuid: "fe908e6232244ca8b51f8799f520e72b" member_type: VOTER last_known_addr { host: "127.11.254.188" port: 35221 } }
I20260812 21:08:13.700237 18850 sys_catalog.cc:455] T 00000000000000000000000000000000 P 08b9af11c5584eea8c0c958f0ced907c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 08b9af11c5584eea8c0c958f0ced907c. Latest consensus state: current_term: 1 leader_uuid: "08b9af11c5584eea8c0c958f0ced907c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08b9af11c5584eea8c0c958f0ced907c" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 42339 } } peers { permanent_uuid: "a08c1a561f454a4cb3c7ac20e1bd179f" member_type: VOTER last_known_addr { host: "127.11.254.189" port: 36639 } } peers { permanent_uuid: "fe908e6232244ca8b51f8799f520e72b" member_type: VOTER last_known_addr { host: "127.11.254.188" port: 35221 } } }
I20260812 21:08:13.700341 18850 sys_catalog.cc:458] T 00000000000000000000000000000000 P 08b9af11c5584eea8c0c958f0ced907c [sys.catalog]: This master's current role is: LEADER
I20260812 21:08:13.700695 18906 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 21:08:13.701144 18906 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 21:08:13.709513 18563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:13.693020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36026 (local address 127.11.254.190:42339)
0812 21:08:13.693150 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:13.693152 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:13.698922 (+ 5770us) server_negotiation.cc:407] Connection header received
0812 21:08:13.699075 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:13.699076 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:13.699086 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:13.699116 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:13.700557 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:13.701257 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:13.702316 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:13.702588 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:13.702627 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:13.704932 (+ 2305us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:08:13.704934 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:08:13.704935 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:08:13.704943 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:08:13.707954 (+ 3011us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:13.708355 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:13.708356 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:13.708356 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:13.708428 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:13.708723 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:13.708724 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:13.708725 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:13.708885 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:08:13.709001 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:13.709322 (+ 321us) server_negotiation.cc:299] Negotiation successful
0812 21:08:13.709420 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103}
I20260812 21:08:13.712641 18699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:13.694725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47112 (local address 127.11.254.189:36639)
0812 21:08:13.698306 (+ 3581us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:13.698308 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:13.703152 (+ 4844us) server_negotiation.cc:407] Connection header received
0812 21:08:13.704276 (+ 1124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:13.704278 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:13.704293 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:13.704327 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:13.705233 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:13.705912 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:13.707205 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:13.707511 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:13.707560 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:13.709339 (+ 1779us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:08:13.709340 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:08:13.709341 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:08:13.709349 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:08:13.711036 (+ 1687us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:13.711548 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:13.711550 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:13.711550 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:13.711622 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:13.711923 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:13.711924 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:13.711924 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:13.712086 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:08:13.712156 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:13.712457 (+ 301us) server_negotiation.cc:299] Negotiation successful
0812 21:08:13.712546 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3549}
I20260812 21:08:13.712805 18836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:13.693527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42690 (local address 127.11.254.188:35221)
0812 21:08:13.693682 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:13.693684 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:13.700771 (+ 7087us) server_negotiation.cc:407] Connection header received
0812 21:08:13.700865 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:13.700866 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:13.700876 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:13.700913 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:13.701634 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:13.702352 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:13.703272 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:13.703582 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:13.703626 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:13.705511 (+ 1885us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:08:13.705513 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:08:13.705513 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:08:13.705533 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:08:13.707240 (+ 1707us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:13.711929 (+ 4689us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:13.711931 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:13.711931 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:13.712004 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:13.712189 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:13.712190 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:13.712191 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:13.712324 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:08:13.712433 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:13.712636 (+ 203us) server_negotiation.cc:299] Negotiation successful
0812 21:08:13.712723 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135}
I20260812 21:08:13.713824 18775 raft_consensus.cc:1261] T 00000000000000000000000000000000 P fe908e6232244ca8b51f8799f520e72b [term 1 FOLLOWER]: Refusing update from remote peer 08b9af11c5584eea8c0c958f0ced907c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 21:08:13.714576 18904 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 08b9af11c5584eea8c0c958f0ced907c [LEADER]: Connected to new peer: Peer: permanent_uuid: "fe908e6232244ca8b51f8799f520e72b" member_type: VOTER last_known_addr { host: "127.11.254.188" port: 35221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 21:08:13.716039 18847 sys_catalog.cc:455] T 00000000000000000000000000000000 P fe908e6232244ca8b51f8799f520e72b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 08b9af11c5584eea8c0c958f0ced907c. Latest consensus state: current_term: 1 leader_uuid: "08b9af11c5584eea8c0c958f0ced907c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08b9af11c5584eea8c0c958f0ced907c" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 42339 } } peers { permanent_uuid: "a08c1a561f454a4cb3c7ac20e1bd179f" member_type: VOTER last_known_addr { host: "127.11.254.189" port: 36639 } } peers { permanent_uuid: "fe908e6232244ca8b51f8799f520e72b" member_type: VOTER last_known_addr { host: "127.11.254.188" port: 35221 } } }
I20260812 21:08:13.716132 18847 sys_catalog.cc:458] T 00000000000000000000000000000000 P fe908e6232244ca8b51f8799f520e72b [sys.catalog]: This master's current role is: FOLLOWER
I20260812 21:08:13.716935 18907 mvcc.cc:205] Tried to move back new op lower bound from 7317786188642463744 to 7317786188593680384. Current Snapshot: MvccSnapshot[applied={T|T < 7317786188642463744}]
I20260812 21:08:13.717020 18850 sys_catalog.cc:455] T 00000000000000000000000000000000 P 08b9af11c5584eea8c0c958f0ced907c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "08b9af11c5584eea8c0c958f0ced907c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08b9af11c5584eea8c0c958f0ced907c" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 42339 } } peers { permanent_uuid: "a08c1a561f454a4cb3c7ac20e1bd179f" member_type: VOTER last_known_addr { host: "127.11.254.189" port: 36639 } } peers { permanent_uuid: "fe908e6232244ca8b51f8799f520e72b" member_type: VOTER last_known_addr { host: "127.11.254.188" port: 35221 } } }
I20260812 21:08:13.717095 18850 sys_catalog.cc:458] T 00000000000000000000000000000000 P 08b9af11c5584eea8c0c958f0ced907c [sys.catalog]: This master's current role is: LEADER
I20260812 21:08:13.717481 18847 sys_catalog.cc:455] T 00000000000000000000000000000000 P fe908e6232244ca8b51f8799f520e72b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "08b9af11c5584eea8c0c958f0ced907c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08b9af11c5584eea8c0c958f0ced907c" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 42339 } } peers { permanent_uuid: "a08c1a561f454a4cb3c7ac20e1bd179f" member_type: VOTER last_known_addr { host: "127.11.254.189" port: 36639 } } peers { permanent_uuid: "fe908e6232244ca8b51f8799f520e72b" member_type: VOTER last_known_addr { host: "127.11.254.188" port: 35221 } } }
I20260812 21:08:13.717571 18847 sys_catalog.cc:458] T 00000000000000000000000000000000 P fe908e6232244ca8b51f8799f520e72b [sys.catalog]: This master's current role is: FOLLOWER
I20260812 21:08:13.717633 18908 mvcc.cc:205] Tried to move back new op lower bound from 7317786188642463744 to 7317786188593680384. Current Snapshot: MvccSnapshot[applied={T|T < 7317786188642463744}]
I20260812 21:08:13.718789 18906 catalog_manager.cc:1347] Generated new cluster ID: 9fc42c39041b43f3adf038b17a3f39a6
I20260812 21:08:13.718843 18906 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 21:08:13.723028 18690 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.254.188" port: 35221 } attempt: 3
Aug 12 21:08:13 dist-test-slave-vpvm krb5kdc[18421](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786568893, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.254.189@KRBTEST.COM for kudu/127.11.254.188@KRBTEST.COM
I20260812 21:08:13.732336 18692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:13.723322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.254.188:35221 (local address 127.0.0.1:42694)
0812 21:08:13.723421 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0812 21:08:13.723433 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 21:08:13.723470 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 21:08:13.723634 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 21:08:13.723634 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 21:08:13.723835 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0812 21:08:13.723887 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:13.723888 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:13.724795 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:13.724797 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:13.725608 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 21:08:13.725611 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 21:08:13.726189 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 21:08:13.726190 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 21:08:13.726263 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:13.726747 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 21:08:13.726752 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 21:08:13.729302 (+ 2550us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 21:08:13.731192 (+ 1890us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:08:13.731193 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:08:13.731194 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:08:13.731424 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:08:13.731648 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 21:08:13.731649 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 21:08:13.731650 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 21:08:13.731715 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 21:08:13.732010 (+ 295us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 21:08:13.732011 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 21:08:13.732078 (+ 67us) client_negotiation.cc:769] Sending connection context
0812 21:08:13.732130 (+ 52us) client_negotiation.cc:240] Negotiation successful
0812 21:08:13.732231 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":69}
I20260812 21:08:13.732926 18836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:13.723407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42694 (local address 127.11.254.188:35221)
0812 21:08:13.723446 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:13.723447 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:13.723457 (+ 10us) server_negotiation.cc:407] Connection header received
0812 21:08:13.723523 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:13.723524 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:13.723533 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:13.723582 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:13.723959 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:13.724692 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:13.725712 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:13.726060 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:13.726114 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:13.729405 (+ 3291us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 21:08:13.729407 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 21:08:13.729407 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 21:08:13.729415 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 21:08:13.731087 (+ 1672us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:13.731507 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:13.731508 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:13.731509 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:13.731580 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 21:08:13.731775 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 21:08:13.731776 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 21:08:13.731776 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 21:08:13.731916 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 21:08:13.731973 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:13.732756 (+ 783us) server_negotiation.cc:299] Negotiation successful
0812 21:08:13.732839 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 21:08:13.734819 18690 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a08c1a561f454a4cb3c7ac20e1bd179f: Bootstrap starting.
I20260812 21:08:13.735549 18690 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a08c1a561f454a4cb3c7ac20e1bd179f: Neither blocks nor log segments found. Creating new log.
I20260812 21:08:13.735821 18690 log.cc:824] T 00000000000000000000000000000000 P a08c1a561f454a4cb3c7ac20e1bd179f: Log is configured to *not* fsync() on all Append() calls
I20260812 21:08:13.743214 18690 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a08c1a561f454a4cb3c7ac20e1bd179f: No bootstrap required, opened a new log
I20260812 21:08:13.746455 18690 raft_consensus.cc:348] T 00000000000000000000000000000000 P a08c1a561f454a4cb3c7ac20e1bd179f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08b9af11c5584eea8c0c958f0ced907c" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 42339 } } peers { permanent_uuid: "a08c1a561f454a4cb3c7ac20e1bd179f" member_type: VOTER last_known_addr { host: "127.11.254.189" port: 36639 } } peers { permanent_uuid: "fe908e6232244ca8b51f8799f520e72b" member_type: VOTER last_known_addr { host: "127.11.254.188" port: 35221 } }
I20260812 21:08:13.746919 18690 raft_consensus.cc:729] T 00000000000000000000000000000000 P a08c1a561f454a4cb3c7ac20e1bd179f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a08c1a561f454a4cb3c7ac20e1bd179f, State: Initialized, Role: FOLLOWER
I20260812 21:08:13.747080 18690 consensus_queue.cc:260] T 00000000000000000000000000000000 P a08c1a561f454a4cb3c7ac20e1bd179f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08b9af11c5584eea8c0c958f0ced907c" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 42339 } } peers { permanent_uuid: "a08c1a561f454a4cb3c7ac20e1bd179f" member_type: VOTER last_known_addr { host: "127.11.254.189" port: 36639 } } peers { permanent_uuid: "fe908e6232244ca8b51f8799f520e72b" member_type: VOTER last_known_addr { host: "127.11.254.188" port: 35221 } }
I20260812 21:08:13.747511 18690 sys_catalog.cc:564] T 00000000000000000000000000000000 P a08c1a561f454a4cb3c7ac20e1bd179f [sys.catalog]: configured and running, proceeding with master startup.
I20260812 21:08:13.747562 18642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "08b9af11c5584eea8c0c958f0ced907c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "a08c1a561f454a4cb3c7ac20e1bd179f"
I20260812 21:08:13.747650 18642 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a08c1a561f454a4cb3c7ac20e1bd179f [term 0 FOLLOWER]: Advancing to term 1
I20260812 21:08:13.748594 18642 raft_consensus.cc:2454] T 00000000000000000000000000000000 P a08c1a561f454a4cb3c7ac20e1bd179f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 08b9af11c5584eea8c0c958f0ced907c in term 1.
I20260812 21:08:13.748826 18640 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a08c1a561f454a4cb3c7ac20e1bd179f [term 1 FOLLOWER]: Refusing update from remote peer 08b9af11c5584eea8c0c958f0ced907c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 21:08:13.747480 18920 sys_catalog.cc:455] T 00000000000000000000000000000000 P a08c1a561f454a4cb3c7ac20e1bd179f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08b9af11c5584eea8c0c958f0ced907c" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 42339 } } peers { permanent_uuid: "a08c1a561f454a4cb3c7ac20e1bd179f" member_type: VOTER last_known_addr { host: "127.11.254.189" port: 36639 } } peers { permanent_uuid: "fe908e6232244ca8b51f8799f520e72b" member_type: VOTER last_known_addr { host: "127.11.254.188" port: 35221 } } }
I20260812 21:08:13.749076 18920 sys_catalog.cc:458] T 00000000000000000000000000000000 P a08c1a561f454a4cb3c7ac20e1bd179f [sys.catalog]: This master's current role is: FOLLOWER
I20260812 21:08:13.749451 18903 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 08b9af11c5584eea8c0c958f0ced907c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a08c1a561f454a4cb3c7ac20e1bd179f" member_type: VOTER last_known_addr { host: "127.11.254.189" port: 36639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 21:08:13.750450 18906 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 21:08:13.750937 18920 sys_catalog.cc:455] T 00000000000000000000000000000000 P a08c1a561f454a4cb3c7ac20e1bd179f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 08b9af11c5584eea8c0c958f0ced907c. Latest consensus state: current_term: 1 leader_uuid: "08b9af11c5584eea8c0c958f0ced907c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08b9af11c5584eea8c0c958f0ced907c" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 42339 } } peers { permanent_uuid: "a08c1a561f454a4cb3c7ac20e1bd179f" member_type: VOTER last_known_addr { host: "127.11.254.189" port: 36639 } } peers { permanent_uuid: "fe908e6232244ca8b51f8799f520e72b" member_type: VOTER last_known_addr { host: "127.11.254.188" port: 35221 } } }
I20260812 21:08:13.751081 18920 sys_catalog.cc:458] T 00000000000000000000000000000000 P a08c1a561f454a4cb3c7ac20e1bd179f [sys.catalog]: This master's current role is: FOLLOWER
I20260812 21:08:13.751157 18920 sys_catalog.cc:455] T 00000000000000000000000000000000 P a08c1a561f454a4cb3c7ac20e1bd179f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "08b9af11c5584eea8c0c958f0ced907c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08b9af11c5584eea8c0c958f0ced907c" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 42339 } } peers { permanent_uuid: "a08c1a561f454a4cb3c7ac20e1bd179f" member_type: VOTER last_known_addr { host: "127.11.254.189" port: 36639 } } peers { permanent_uuid: "fe908e6232244ca8b51f8799f520e72b" member_type: VOTER last_known_addr { host: "127.11.254.188" port: 35221 } } }
I20260812 21:08:13.751215 18906 catalog_manager.cc:1504] Loading token signing keys...
I20260812 21:08:13.751274 18920 sys_catalog.cc:458] T 00000000000000000000000000000000 P a08c1a561f454a4cb3c7ac20e1bd179f [sys.catalog]: This master's current role is: FOLLOWER
I20260812 21:08:13.752679 18944 mvcc.cc:205] Tried to move back new op lower bound from 7317786188776681472 to 7317786188593680384. Current Snapshot: MvccSnapshot[applied={T|T < 7317786188642463744}]
I20260812 21:08:13.756016 18903 sys_catalog.cc:455] T 00000000000000000000000000000000 P 08b9af11c5584eea8c0c958f0ced907c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "08b9af11c5584eea8c0c958f0ced907c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08b9af11c5584eea8c0c958f0ced907c" member_type: VOTER last_known_addr { host: "127.11.254.190" port: 42339 } } peers { permanent_uuid: "a08c1a561f454a4cb3c7ac20e1bd179f" member_type: VOTER last_known_addr { host: "127.11.254.189" port: 36639 } } peers { permanent_uuid: "fe908e6232244ca8b51f8799f520e72b" member_type: VOTER last_known_addr { host: "127.11.254.188" port: 35221 } } }
I20260812 21:08:13.756114 18903 sys_catalog.cc:458] T 00000000000000000000000000000000 P 08b9af11c5584eea8c0c958f0ced907c [sys.catalog]: This master's current role is: LEADER
I20260812 21:08:13.757584 18906 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 08b9af11c5584eea8c0c958f0ced907c: Generated new TSK 0
I20260812 21:08:13.757762 18906 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 21:08:13.758673 18950 catalog_manager.cc:1259] Loaded cluster ID: 9fc42c39041b43f3adf038b17a3f39a6
I20260812 21:08:13.758754 18950 catalog_manager.cc:1552] T 00000000000000000000000000000000 P a08c1a561f454a4cb3c7ac20e1bd179f: loading cluster ID for follower catalog manager: success
I20260812 21:08:13.759872 18950 catalog_manager.cc:1574] T 00000000000000000000000000000000 P a08c1a561f454a4cb3c7ac20e1bd179f: acquiring CA information for follower catalog manager: success
I20260812 21:08:13.760207 18950 catalog_manager.cc:1602] T 00000000000000000000000000000000 P a08c1a561f454a4cb3c7ac20e1bd179f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260812 21:08:13.777632 18699 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:08:13.776137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47120 (local address 127.11.254.189:36639)
0812 21:08:13.776185 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:13.776187 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:13.776912 (+ 725us) server_negotiation.cc:407] Connection header received
0812 21:08:13.777036 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:13.777038 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:13.777051 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:13.777082 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:13.777556 (+ 474us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47120: BlockingRecv error: recv got EOF from 127.0.0.1:47120 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260812 21:08:13.778465 18563 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:08:13.776590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36042 (local address 127.11.254.190:42339)
0812 21:08:13.776635 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:13.776637 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:13.777889 (+ 1252us) server_negotiation.cc:407] Connection header received
0812 21:08:13.778024 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:13.778026 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:13.778039 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:13.778068 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:13.778393 (+ 325us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36042: BlockingRecv error: recv got EOF from 127.0.0.1:36042 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260812 21:08:13.779531 18836 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:08:13.778840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42710 (local address 127.11.254.188:35221)
0812 21:08:13.778880 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:13.778882 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:13.779021 (+ 139us) server_negotiation.cc:407] Connection header received
0812 21:08:13.779112 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:13.779114 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:13.779124 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:13.779151 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:13.779465 (+ 314us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42710: BlockingRecv error: recv got EOF from 127.0.0.1:42710 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 21:08:13.784802 18699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:13.781207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47132 (local address 127.11.254.189:36639)
0812 21:08:13.781259 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:13.781261 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:13.781393 (+ 132us) server_negotiation.cc:407] Connection header received
0812 21:08:13.781505 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:13.781506 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:13.781557 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:13.781602 (+ 45us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 21:08:13.781804 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:13.782632 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:13.783819 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:13.784140 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:13.784177 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:13.784404 (+ 227us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 21:08:13.784542 (+ 138us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 21:08:13.784592 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:13.784701 (+ 109us) server_negotiation.cc:299] Negotiation successful
0812 21:08:13.784722 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
W20260812 21:08:13.788269 18836 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:08:13.787463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42724 (local address 127.11.254.188:35221)
0812 21:08:13.787504 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:13.787506 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:13.787673 (+ 167us) server_negotiation.cc:407] Connection header received
0812 21:08:13.787778 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:13.787779 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:13.787799 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:13.787828 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:13.788215 (+ 387us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42724: BlockingRecv error: recv got EOF from 127.0.0.1:42724 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 21:08:13.794193 18699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:13.790799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47142 (local address 127.11.254.189:36639)
0812 21:08:13.790837 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:13.790838 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:13.791109 (+ 271us) server_negotiation.cc:407] Connection header received
0812 21:08:13.791201 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:13.791202 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:13.791213 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:13.791240 (+ 27us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 21:08:13.791413 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:13.792189 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:13.793334 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:13.793631 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:13.793670 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:13.793857 (+ 187us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 21:08:13.793893 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 21:08:13.793918 (+ 25us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:13.794087 (+ 169us) server_negotiation.cc:299] Negotiation successful
0812 21:08:13.794112 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
W20260812 21:08:13.797432 18836 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:08:13.796680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42732 (local address 127.11.254.188:35221)
0812 21:08:13.796721 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:13.796723 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:13.796900 (+ 177us) server_negotiation.cc:407] Connection header received
0812 21:08:13.797007 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:13.797008 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:13.797020 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:13.797047 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:13.797376 (+ 329us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42732: BlockingRecv error: recv got EOF from 127.0.0.1:42732 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 21:08:13.803819 18699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:13.800524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47148 (local address 127.11.254.189:36639)
0812 21:08:13.800571 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:13.800573 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:13.800809 (+ 236us) server_negotiation.cc:407] Connection header received
0812 21:08:13.800897 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:13.800899 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:13.800909 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:13.800934 (+ 25us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 21:08:13.801119 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:13.801826 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:13.803003 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:13.803333 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:13.803372 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:13.803567 (+ 195us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 21:08:13.803604 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 21:08:13.803629 (+ 25us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:13.803724 (+ 95us) server_negotiation.cc:299] Negotiation successful
0812 21:08:13.803742 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
W20260812 21:08:13.806993 18836 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:08:13.806141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42748 (local address 127.11.254.188:35221)
0812 21:08:13.806181 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:13.806182 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:13.806452 (+ 270us) server_negotiation.cc:407] Connection header received
0812 21:08:13.806547 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:13.806548 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:13.806561 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:13.806587 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:13.806923 (+ 336us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42748: BlockingRecv error: recv got EOF from 127.0.0.1:42748 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 21:08:13.815848 18699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:13.812275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47150 (local address 127.11.254.189:36639)
0812 21:08:13.812322 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:13.812324 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:13.812609 (+ 285us) server_negotiation.cc:407] Connection header received
0812 21:08:13.812714 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:13.812715 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:13.812726 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:13.812753 (+ 27us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 21:08:13.812960 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:13.813716 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:13.814999 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:13.815308 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:13.815347 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:13.815581 (+ 234us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 21:08:13.815618 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 21:08:13.815644 (+ 26us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:13.815741 (+ 97us) server_negotiation.cc:299] Negotiation successful
0812 21:08:13.815763 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
W20260812 21:08:13.819528 18836 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:08:13.818688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42758 (local address 127.11.254.188:35221)
0812 21:08:13.818748 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:13.818750 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:13.818936 (+ 186us) server_negotiation.cc:407] Connection header received
0812 21:08:13.819064 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:13.819065 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:13.819077 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:13.819104 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:13.819468 (+ 364us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42758: BlockingRecv error: recv got EOF from 127.0.0.1:42758 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
I20260812 21:08:13.832332 18699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:13.828815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47152 (local address 127.11.254.189:36639)
0812 21:08:13.828874 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:13.828876 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:13.829163 (+ 287us) server_negotiation.cc:407] Connection header received
0812 21:08:13.829266 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:13.829267 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:13.829278 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:13.829305 (+ 27us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 21:08:13.829487 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:13.830356 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:13.831483 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:13.831783 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:13.831820 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:13.832025 (+ 205us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 21:08:13.832058 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 21:08:13.832083 (+ 25us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:13.832227 (+ 144us) server_negotiation.cc:299] Negotiation successful
0812 21:08:13.832249 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
W20260812 21:08:13.835728 18836 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:08:13.834965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42766 (local address 127.11.254.188:35221)
0812 21:08:13.835003 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:13.835005 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:13.835185 (+ 180us) server_negotiation.cc:407] Connection header received
0812 21:08:13.835277 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:13.835278 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:13.835288 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:13.835316 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:13.835672 (+ 356us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42766: BlockingRecv error: recv got EOF from 127.0.0.1:42766 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
I20260812 21:08:13.857021 18699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:13.853247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47158 (local address 127.11.254.189:36639)
0812 21:08:13.853295 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:13.853297 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:13.853665 (+ 368us) server_negotiation.cc:407] Connection header received
0812 21:08:13.853776 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:13.853778 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:13.853789 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:13.853820 (+ 31us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 21:08:13.854100 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:13.854811 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:13.855939 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:13.856350 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:13.856410 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:13.856706 (+ 296us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 21:08:13.856746 (+ 40us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 21:08:13.856776 (+ 30us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:13.856889 (+ 113us) server_negotiation.cc:299] Negotiation successful
0812 21:08:13.856913 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
W20260812 21:08:13.860533 18836 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:08:13.859665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42770 (local address 127.11.254.188:35221)
0812 21:08:13.859713 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:13.859715 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:13.859914 (+ 199us) server_negotiation.cc:407] Connection header received
0812 21:08:13.860029 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:13.860030 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:13.860042 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:13.860074 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:13.860473 (+ 399us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42770: BlockingRecv error: recv got EOF from 127.0.0.1:42770 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 21:08:13.897969 18699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:13.895257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47164 (local address 127.11.254.189:36639)
0812 21:08:13.895311 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:13.895314 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:13.895404 (+ 90us) server_negotiation.cc:407] Connection header received
0812 21:08:13.895528 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:13.895528 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:13.895537 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:13.895589 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 21:08:13.895742 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:13.896233 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:13.897330 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:13.897525 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:13.897574 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:13.897740 (+ 166us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 21:08:13.897767 (+ 27us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 21:08:13.897798 (+ 31us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:13.897868 (+ 70us) server_negotiation.cc:299] Negotiation successful
0812 21:08:13.897884 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
W20260812 21:08:13.900703 18836 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:08:13.899848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42780 (local address 127.11.254.188:35221)
0812 21:08:13.899894 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:13.899895 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:13.900213 (+ 318us) server_negotiation.cc:407] Connection header received
0812 21:08:13.900250 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:13.900251 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:13.900262 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:13.900310 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:13.900645 (+ 335us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42780: BlockingRecv error: recv got EOF from 127.0.0.1:42780 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 21:08:13.968665 18699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:13.966048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47180 (local address 127.11.254.189:36639)
0812 21:08:13.966121 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:13.966123 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:13.966243 (+ 120us) server_negotiation.cc:407] Connection header received
0812 21:08:13.966327 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:13.966328 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:13.966343 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:13.966399 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 21:08:13.966613 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:13.967131 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:13.967977 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:13.968174 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:13.968241 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:13.968400 (+ 159us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 21:08:13.968438 (+ 38us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 21:08:13.968487 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:13.968551 (+ 64us) server_negotiation.cc:299] Negotiation successful
0812 21:08:13.968576 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
W20260812 21:08:13.971395 18836 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:08:13.970674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42784 (local address 127.11.254.188:35221)
0812 21:08:13.970739 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:13.970742 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:13.970789 (+ 47us) server_negotiation.cc:407] Connection header received
0812 21:08:13.970910 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:13.970911 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:13.970923 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:13.970975 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:13.971328 (+ 353us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42784: BlockingRecv error: recv got EOF from 127.0.0.1:42784 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
I20260812 21:08:14.102785 18699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:14.100618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47196 (local address 127.11.254.189:36639)
0812 21:08:14.100677 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:14.100678 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:14.100727 (+ 49us) server_negotiation.cc:407] Connection header received
0812 21:08:14.100800 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:14.100801 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:14.100811 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:14.100854 (+ 43us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 21:08:14.100962 (+ 108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:14.101461 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:14.102247 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:14.102436 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:14.102490 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:14.102600 (+ 110us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 21:08:14.102630 (+ 30us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 21:08:14.102666 (+ 36us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:14.102702 (+ 36us) server_negotiation.cc:299] Negotiation successful
0812 21:08:14.102720 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
W20260812 21:08:14.105047 18836 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:08:14.104443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42800 (local address 127.11.254.188:35221)
0812 21:08:14.104508 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:14.104510 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:14.104518 (+ 8us) server_negotiation.cc:407] Connection header received
0812 21:08:14.104606 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:14.104607 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:14.104617 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:14.104660 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:14.104972 (+ 312us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42800: BlockingRecv error: recv got EOF from 127.0.0.1:42800 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
I20260812 21:08:14.364794 18699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:14.362361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47212 (local address 127.11.254.189:36639)
0812 21:08:14.362425 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:14.362427 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:14.362528 (+ 101us) server_negotiation.cc:407] Connection header received
0812 21:08:14.362565 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:14.362566 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:14.362577 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:14.362636 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 21:08:14.362751 (+ 115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:14.363255 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:14.364047 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:14.364246 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:14.364314 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:14.364495 (+ 181us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 21:08:14.364530 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 21:08:14.364577 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:14.364676 (+ 99us) server_negotiation.cc:299] Negotiation successful
0812 21:08:14.364711 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
W20260812 21:08:14.367197 18836 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 21:08:14.366642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42808 (local address 127.11.254.188:35221)
0812 21:08:14.366714 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:14.366715 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:14.366724 (+ 9us) server_negotiation.cc:407] Connection header received
0812 21:08:14.366775 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:14.366776 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:14.366786 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:14.366828 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0812 21:08:14.367139 (+ 311us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42808: BlockingRecv error: recv got EOF from 127.0.0.1:42808 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 21:08:14.698877 18888 catalog_manager.cc:1259] Loaded cluster ID: 9fc42c39041b43f3adf038b17a3f39a6
I20260812 21:08:14.698959 18888 catalog_manager.cc:1552] T 00000000000000000000000000000000 P fe908e6232244ca8b51f8799f520e72b: loading cluster ID for follower catalog manager: success
I20260812 21:08:14.699821 18888 catalog_manager.cc:1574] T 00000000000000000000000000000000 P fe908e6232244ca8b51f8799f520e72b: acquiring CA information for follower catalog manager: success
I20260812 21:08:14.700141 18888 catalog_manager.cc:1602] T 00000000000000000000000000000000 P fe908e6232244ca8b51f8799f520e72b: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 21:08:14.882867 19198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:14.880368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47222 (local address 127.11.254.189:36639)
0812 21:08:14.880502 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:14.880504 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:14.880580 (+ 76us) server_negotiation.cc:407] Connection header received
0812 21:08:14.880640 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:14.880641 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:14.880652 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:14.880713 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 21:08:14.880823 (+ 110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:14.881337 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:14.882070 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:14.882284 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:14.882349 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:14.882529 (+ 180us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 21:08:14.882570 (+ 41us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 21:08:14.882631 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:14.882746 (+ 115us) server_negotiation.cc:299] Negotiation successful
0812 21:08:14.882773 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":64,"threads_started":1}
I20260812 21:08:14.887162 19207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:14.884596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42810 (local address 127.11.254.188:35221)
0812 21:08:14.884725 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:14.884728 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:14.884780 (+ 52us) server_negotiation.cc:407] Connection header received
0812 21:08:14.884846 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:14.884847 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:14.884924 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:14.884993 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 21:08:14.885109 (+ 116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:14.885612 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:14.886374 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:14.886586 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:14.886649 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:14.886802 (+ 153us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 21:08:14.886928 (+ 126us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 21:08:14.886999 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:14.887045 (+ 46us) server_negotiation.cc:299] Negotiation successful
0812 21:08:14.887069 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106,"thread_start_us":61,"threads_started":1}
I20260812 21:08:14.892113 19207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:14.889000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42812 (local address 127.11.254.188:35221)
0812 21:08:14.889040 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:14.889042 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:14.889327 (+ 285us) server_negotiation.cc:407] Connection header received
0812 21:08:14.889355 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:14.889356 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:14.889366 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:14.889413 (+ 47us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 21:08:14.889555 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:14.890284 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:14.891276 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:14.891481 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:14.891536 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:14.891704 (+ 168us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 21:08:14.891743 (+ 39us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 21:08:14.891794 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:14.891995 (+ 201us) server_negotiation.cc:299] Negotiation successful
0812 21:08:14.892024 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260812 21:08:14.892202 19198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:14.889129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47226 (local address 127.11.254.189:36639)
0812 21:08:14.889176 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:14.889178 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:14.889303 (+ 125us) server_negotiation.cc:407] Connection header received
0812 21:08:14.889442 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:14.889444 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:14.889463 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:14.889538 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 21:08:14.889642 (+ 104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:14.890245 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:14.891456 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:14.891743 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:14.891799 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:14.891917 (+ 118us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 21:08:14.891952 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 21:08:14.891994 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:14.892092 (+ 98us) server_negotiation.cc:299] Negotiation successful
0812 21:08:14.892127 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260812 21:08:14.892616 19219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 21:08:14.889014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36050 (local address 127.11.254.190:42339)
0812 21:08:14.889271 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0812 21:08:14.889273 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 21:08:14.889281 (+ 8us) server_negotiation.cc:407] Connection header received
0812 21:08:14.889343 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 21:08:14.889344 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 21:08:14.889518 (+ 174us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 21:08:14.889607 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 21:08:14.889805 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:14.890561 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:14.891583 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 21:08:14.891894 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 21:08:14.891951 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 21:08:14.892157 (+ 206us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 21:08:14.892350 (+ 193us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 21:08:14.892415 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0812 21:08:14.892500 (+ 85us) server_negotiation.cc:299] Negotiation successful
0812 21:08:14.892530 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":236,"thread_start_us":62,"threads_started":1}
I20260812 21:08:14.894251 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 18443
I20260812 21:08:14.916355 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 18571
I20260812 21:08:14.929087 12282 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskymWbKN/build/release/bin/kudu with pid 18705
2026-08-12T21:08:14Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3047 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (6020 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (287578 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST